Automation Bug Postmortem

This was certainly fun to write.  I tried to keep it as accessible as possible for non-programmers, but there’s a section at the bottom for those interested in the exact details.

A brief refresher: One operation absolutely essential for QTTabBar to function is hit testing, which means “figuring out what file or folder the mouse is over”. This used to be super easy, but with the advent of the new ItemsView control, it’s no longer trivial. Fortunately, it’s still possible through the use of the UI Automation library. Unfortunately, the ItemsView has a nasty bug in it: every time you make a UI Automation query, something builds up inside the control that introduces a tiny bit of lag, permanently. That means every time I ask what file the mouse is over (which I need to do very often) the lag grows a tiny bit. After many queries, the lag becomes very noticeable, and after a while it becomes excruciating. The lag goes away when Explorer is restarted, so users that restart their computers often probably barely notice this problem at all. But for power users who infrequently restart, or laptop users who always hibernate, it’s a major issue.

My quest to find a workaround would have been completely impossible where it not for Microsoft’s Public Symbol Files. Microsoft supplies debugging files for all important executable files in Windows. These files don’t give you the source code, of course, but they do give you the names and locations of all the internal functions. This was enough for me to attach a debugger to Explorer and (after a lot of digging through assembly code) figure out exactly the bug that was causing the problem.

Every time a UI Automation query is issued, the connection to the ItemsView’s automation object is created anew. Whenever it’s created, the UI Automation library registers for certain event notifications from the ItemsView, such as “notify me when the selection changes,” and other such events. It’s during this event registration process that the lag increases. It seems a little wasteful to create and destroy the connection every time a single hit test query is executed, and indeed, my first idea was to somehow cache it, so that it only needed to be created once. But over the course of trying to do that (which turned out to be futile anyway), I noticed something very critical: before registering the events, the ItemsView automation object is first asked if it is the type of object for which events can be registered. If the object were to reply, “no, I’m not that kind of object,” then the code introducing the lag is skipped right over! Figuring out how to intercept that request and respond with a fake reply was tricky, but once I did, the lag vanished.

Of course, this does introduce another problem: if any UI Automation application was actually counting on those event notifications, then that application wouldn’t get them once my hook was in place. QTTabBar does not use these events, since they are extremely inefficient, and there are better ways of getting the necessary notifications. And honestly, if anyone else is using these events, they shouldn’t be, specifically because of this bug! So, I’m not too worried about that. In the worst case, if I become aware that it is indeed causing a major problem with some other application, I could probably figure out a way to block the event registration only for QTTabBar’s requests, and no others.

Here’s a more technical explanation, for those that found the above explanation lacking in detail. The main function I need for hit testing is IUIAutomation.ElementFromPoint function, which retrieves the element under the mouse. After a long time of stepping through the assembly, I found that the lag was occurring because Automation events were being registered and never unregistered. This registration occurs during the call to UiaReturnRawElementProvider, which oh-so-conveniently includes a pointer to the ItemsView’s IRawElementProviderSimple as a parameter. I saw in the assembly that the element provider’s QueryInterface method is called to get access to its IRawElementProviderAdviseEvents interface, and from there the events are registered. Ahah! I knew immediately that if they were calling QueryInterface, then they will dutifully check the return code to make sure it succeeded. Sure enough, if QueryInterface doesn’t return S_OK, the event registration section is cleanly skipped, without any the loss of any other functionality. Much as I like to rag on Microsoft, I have never been more thankful for their excellent coding practice. It saved us here!

So, with the ever-amazing MinHook library, I placed an API hook on UiaReturnRawElementProvider, which I used to place another hook on the ItemsView’s IRawElementProviderSimple .QueryInterface. Then I can filter out those calls that are asking for IRawElementProviderAdviseEvents and spoof a return of E_NOINTERFACE.

So in the end, it’s pretty close to the best case scenario. The lag is completely fixed. Hit tests are free again, using a documented method that is unlikely to break in Windows 8. I can finally get rid of all that horrible spaghetti code I was using to minimize the number of hit test operations. And even more importantly, I can incorporate a hit test function into the plugin library, so that anyone who wants to write something that requires hit testing can use it.

This whole ordeal has really been a huge learning experience for me. I’m going to back some other problems I was facing with the ItemsView and saying “oh, that’s easy now that I know how to do this…” And I still feel like I’m just scratching the surface of what’s possible. Figuring out how to bend Explorer to my will is such an addictive puzzle. People sometimes ask me why I submit myself to dealing with Microsoft’s problems instead of just making my own file browser from scratch. There are, of course, some real answers to that question, but my first response is “Because this way is so much more fun!”

Thanks for reading. Beta 2 is on the way!

46 comments to Automation Bug Postmortem

  • Kerry Johnson

    The perfectionist is at it again! Thanks for taking the time in hunting down these bugs and sharing your finds/solution.

    Beta 2 is most definitely going to be welcome. I put my laptop to sleep on the weekdays and notice the lag start to build up in the latter end of the week. But I usually restart sometime on the weekend.

  • bluefat

    i am very like this tool.i have used in win7 32bit. but i can’t

    regular use in Server 2008 R2.you can test and found where’s the bug,thank you!

  • teroan

    Although i’ve not had any problems with the current version after i used the “hack” to get the old list view back. It’s good to see work being done to get this “new and improved” ItemsView working properly with QTTabBar.

    Thanks you!

  • Rob

    Always good to read someones enthusiasm for a project, even if i don’t grasp the full implications of it all.
    I do understand the need to solve problems though, and good to see your efforts pay off.
    Nice one :)

  • Nicu

    Love the project and I like the way you keep us up to date with details and explanations.
    This kind of mentality and dedication I saw only on VirtualDub project.
    Thank u so much.

  • Mike Connor

    Well done. Sounds very exciting. Looking forward to the new releases with barely contained anticipation.

    Regards….Mike Connor

  • tuxplorer

    Hopefully, the technical details should help Microsoft fix the bug if they care enough.

  • Machete

    all hail SHININGMASAMUNE!

  • albert

    Well done perfectionist! You have fun creating QTTabBar and we have fun when using it. Thanks a lot and look forward to new release.

  • Michael

    When you took over the project, you initially stated your admiration and impression for Quizos code-quality.

    I think the code found a worthy successor!

    Your always interesting posts are showing a dedication which is nothing less then a joy to see!

    Michael

  • Take a look to windowtabs http://www.windowtabs.com/
    it is faster and most beautiful than qttabbar…
    you could take a leaf…
    (it’s a simple suggestion from a qttabbar user)

  • Taras

    Yes Windowtabs looks good but it is addressing the completely different problems. While some overlap may indeed exist.

    I put is simple: Windowtabs (WT) is a Desktop management utility and QTTabBar (QTB) is a file management one. Thus WT just cannot be some how better or worse than QTB as they do not even compete.

    And I am not even talking about technical differences (e.g. extensibility with add-ins).

  • Mesonto

    Thank you for all your hard work on QTTabBar. I have been using your betas on Win7 64bit since I installed it. It has completely changed the way in which I work in Explorer.

    I also looked at Carlo’s version of ubiquitous windows tabs and he has an interedting take on them as well. (although not at all what I want)

    I am writing this because I am wondering (and I have seen other people request this) can you possibly make an extreme light version of this tool?

    QTTabBarEtremeLite for people who only want the tabs and nothing more. I would really appreciate a one function “tab only” version of what you have done. It would of course use less resources, it may be quicker, less prone to crashing, and easier to work with. In the end all I need (and many others I have introduced this app to) are Win Explorer tabs.

    However in the end and if you don’t do this, I don’t know what I would do without this added functionality that MS should have built in.

    Many thanks again for a serious application that changes the way I work with my computer.

  • Let me be clear … I would not say that WT is better than QT …
    I just suggest to get the best that WT has (look and feel in primis).
    QT remain the best filemanager on windows (for me)…

    • Taras

      >[Carlo April 5]
      Take a look to windowtabs … it is faster and most beautiful than qttabbar…

      >[Carlo April 8]
      Let me be clear … I would not say that WT is better than QT

      It is too confusing to me… :o )

  • xeon

    Been meaning to comment here, just wanted to say I recently decided to see how the new QTTabBar incarnation is coming along.. using it now with Win7 x64 SP1 (put off using vis7a as a default OS for a long time, though I’ve used the original QTTabBar for XPsp3 for many years now along with other explorer shell extensions that didn’t make it past the vista dumbification screwup. Foldersize, and DMEXbar etc etc etc :( ((

    Your work in fixing and enhancing the crap vis7a MS file explorer shell beyond commendable, enjoy reading your comments on the lengths you have to goto towards fixing and working around MS code etc lol… only tried your QTTabBar version early june last year with Win7 in VM, but wasn’t enough to move me, looks like Win7 has finally matured with a decent number of shell extensions to pretty much bridge most the gaps of lame development the explorer shellteam seems to design explorer for… had to get Classic Shell for the rest of the enhancements.. like quick buttons for details/thumb view.. and to fix a number of other issues including getting the classic statusbar.

    Anyway wanted to know if you were aware of why the QTTabBar ‘Search box’ (another awesome feature of QTTabBar) is slower in Win7 compared to XP.. I mean in terms of entering/changing text in its search field, and the explorer detail/listview filtering out files/folders… its like a display delay, hard to describe, lets just say it isn’t instant.. and seeing the scrollbar move… basically compared to XP its slow and not at all instant like it used to be.

    Another issue I have always had with Vis7a is the fullrow selection noobtardation… it can be disabled, but that only goes to full ‘column’ row selection.. which I can barely live with… as its still annoying (I could describe the intricate details of why the previous method is better but I’ll just get on with asking if there is anychance of the classic and vastly better only upto ‘text length row’ file/folder selection being hacked in?

    Option to have a QTTabBar toggle button/or dialog option created, that can enable/disable the useless and barely customizable crap iconless CommandBar abomination? It’s just a waste of toolbar height.. though I suppose if QTTabBar toolbar/tabs could actually utilize the CommandBar, provide a way to arrange/remove/add what is already on it, it would allow me to hide the Menu Bar, though its not so bad, basically the Menubar is just 1 row, with only the ‘File’ item showing, the rest are hidden in the >> drop menu as I shifted QTTabBar right up alongside it.

    Quizzo started the QTAddressBar.. it never got far unfortunately, have you thought about taking it on and providing more customizable options.. always disliked the total lack of customization for the vis7a address/breadcrumb bar.. like padding distance of crumbs etc…would be nice if QTTabBar (can show navigation buttons, yet has no address bar) +QTAddressBar ..could just have a toolbar of its own (well it would still be a toolbar so other toolbar extensions could go on it like Classic Shell etc. basically so that all defaulty and barely if at all customizable MS crap can be removed off the explorer file shell window, and something more streamlined put in place instead…

    Maybe its possible to have QTTabBar TABS on the same row as the MS address/crumb bar? (I have the desktop file Search removed, but it seems MS have that entire area locked and very uncustomizable, just wondering..

    oh Qttabbar has a search button that opens a new explorer window with the rubbish Win7 desktop file index search.. any chance of hooking that into calling the search program ‘Everything’ by voidtools.com instead?

    Windows 7 Explorer’s Navigation Pane:

    Favorites

    Libraries

    Computer

    Network

    Control Panel

    I know its possible to add/remove these above shortcuts.. but seems to be no way of re-arranging them (Like having ‘Computer’ at the top)… anything insight into that being possible?

  • xeon

    “Dragging Explorer into the modern era” :)

    So true.. QTTabBar rocks… “Up one level when double clicking in an empty area of a folder ” For the W1N!!!! its the best shortcut provided by QTTabBar, well at least after disabling stupid fecking full row noob detail selection that is.

  • Boki

    Very nice explorer plugin, if not the best one.

    As a suggestion, could you add a toolbar button to access the QTTabBar options?

  • RudeBoySes

    WindowsTabs?

    haha .. please!

    that crap sucks.. it initiates tabs for any windows/app that you launch, causing more resources to be used.
    and they want to charge? gTFO!

    i like QTTabBar because i don’t have to use a separate app.
    i like Windows Explorer.
    so i want a seamless integration, and seamless operation.
    which QTTabBar does.

    Honestly, the better way is to ask for donations.
    Rather than demand.
    If you want an app. to be popular, do what QTTabBar does.

    Make sure your design is well thought out, Make it freeware and ask for donations.

    F.Y.I
    i am willing to help test any future betas.
    Im running Windows 7 SP1 64-bit.
    My Email: RudeboySes@live.com

  • xeon

    The ‘Shortcut Keys’ list doesn’t show anything for ‘Up one level’ …yes you can set it QTTabBar ‘Tab’ settings for like double click.. but if you want to use say Strokeit to quickly gesture and have it send a hotkey to the explorer frame to go up a directory.. there isn’t any hotkey to assign that same function. Double clicking the background of an explorer frame is so stupid with winshi7e explorer.

    Mainly because of this issue:
    “Disabling full row select without using the old listview is completely impossible (or at least, would require far far more work than I have time to do). Disabling the command bar doesn’t look possible either. Sorry…”

    it’s so stupid that you can double click on the ‘name column’ row, and it will open said target file/fodler.. yet if you want to drag and drop said file/folder target to another application etc.. it won’t do shit unless you actual start the drag process from the ‘ACTUAL File/folder name’ aka how a decent file manager works …obviously this is so that you can still marquee select file folders starting from the same mouseover area of the ‘name column’ where file/folders get that highlighting.. but jeeesh it is so fucking stupid.. it also means that unlike marque selecting in xp explorer.. you can quickly selecting file/folders with longer file names, while not having the selection mask grabbing file/folders with shorter filename titles.. honestly I could strangle the morons at MS who broke that shit for something so fucking dumb and inferior (wouldn’t mind if it were optional workflow aesthetic), surely I can’t be the person who thinks its stupid.

    And I use the ‘Up one level’ functionality mainly because it selects the previously visited folder, making it quicker to say delete that folder without having to search for it. Where as if you ‘go back’ in history the defaulty functionality of shit explorer doesn’t select the folder that you were just browsing (I mean intelligently obviously) ie if it saw that you were in “c:\winshi7e\microsuck” and browsed back in history a directory, and that folder history took you to c:\winshi7e, then it should select/highlight the folder “microsuck”… obviously if the back history jumped to a different folder/drive with the same “microsuck” folder then it should ‘intelligently’ know that isn’t the same folder you were in previously, by the fact the root drive might not be the same for one. Only it doesn’t, personally I’d rather it did select the folder than not at all. Such logical and enhanced workflow functionality escapes MS.

    “I have some plans for the breadcrumb bar; middle-click = new tab is coming in Beta 2!”

    sounds good/somewhat useful.. don’t suppose drag & dropping files/folder onto the breadcrumb bar could ‘move’ said target(s) to that address location of the breadcrumb? just a thought, though I suppose if middle click works then its only added few steps to drag/drop files into the newly opened tab with that new functionality :) MS is probably dumbificating Explorer right now with a shitty ribbon bar, while actual useful power user file workflow enhancements like improving the breadcrumb bar are totally sidelined.

    “You can disable or autohide the command bar:

    http://www.askvg.com/how-to-make-folder-band-auto-hidden-in-windows-vista/

    Yup that works great thanks.. I think I had seen it before, but maybe it was a full disable method which looked like to much messing around if you wanted it back on again for some of its burried menu settings, like a said in my previous comment I just wanted a quicker way of toggling it on/off directly from the QTTabBar ui like button. Still this autohide method does work, if not a little buggy if you don’t place the cursor in the right position to make it show up. Looks like Win8 still has the same barely customizable shit and useless looking command bar.

    “Search box slowness is probably the same issue as the scrolling lag. That said, it could still be even faster. I’ll be improving the search functionality a little later.”

    I hope you can look into it, had a folder with 4000 files, and using the qttabbar in frame search was just painfully slow…watching the files disappear and seeing the scrollbar lag slowly expand and move… another thing is it would be nice if double clicking the search text box, deleted the search text.. or maybe a little red cross to clear the text.. and that is another thing.. just removing the search text causes it to go slow just populating the folder frame with files again.. I mean honestly it would be faster to go back out that folder and come back into again like you were just opening the same folder as the files show instantly.

    thanks for reading my rants :)

  • xeon

    “it won’t do shit unless you actual start the drag process from the ‘ACTUAL File/folder TITLE name’ ”

    Forgot to stress teh importance of the file/folder title, and not the name column row highlighting area field, just incase anyone was confused by that.

  • xeon

    DMEXbar had some great explorer shell enhancements and the whole thing was customizable so it worked nicely alongside QTTabBar on xp..sadly the original developer never ported it to vis7a can’t remember what his words were now exactly :) … the reason I bring it up is that it had a great multi-file renaming power utility built in which out of all its useful functionality is the most sorely missed thing.

    Don’t suppose any chance of Qttabbar picking up on any of the features it had..

    http://www.freewaregenius.com/2008/02/06/dmexbar-brings-the-functionality-of-a-dual-pane-file-manager-to-windows-explorer/

    the link provides at least some overview of what it was capable of. Unfortunately reviewer failed to mention a number of features/options including multi-file renaming utility. Still if anyone has XP/or vm to try it or can provide some alternative small utility apps that might be worth checking out that would be good (I really hate explorer file manager replacements as they are mainly all shit, and never really are worthy replacements for explorer as a filemanager, at least with useful shell extensions like qttabbar (as every opened folder can be a floating window where the taskbar is pretty much a tabbar for opened folder windows) so please don’t mention any of them I tried them all and neverr bother with them.. just wanting a file rename utility that I can setup with Qttabbar ‘Applications’ so I can pass selected files to suggested utility would be good. Preferably one as good as dmexbar and not totally bloated.

  • xeon

    Well I’m shitting on microsuck just thought I’d point this thread out…

    http://social.technet.microsoft.com/Forums/en-US/itprovistadesktopui/thread/b2aff758-8a79-4889-9cc5-e33c88fae664/

    The XP folder thumbnail icons were way better.. the Windows 7/vista folder thumbnails are beyond crap, as many in that thread have pointed out, its like the feedback on win7 explorer went out the window.. and vista dumbification was okay.

    I’ve tried looking for some way of hacking in the originals, but no luck, it would be amazing if QTTabBar found a way changing the shell dll from where this stupid vista angled folder thumbnail layout resides. I hold no hope on it, just putting another message out the internets and maybe enough feedback about will result in a future improvement ie back to how it used to be when it were better.

  • This forum needed sahnkig up and you’ve just done that. Great post!

  • Didn’t know the forum rules allowed such brlilinat posts.

  • CB1VoQ , [url=http://qwhotqxtxahk.com/]qwhotqxtxahk[/url], [link=http://lzwoebrogqds.com/]lzwoebrogqds[/link], http://obqxreqmacgj.com/

  • HHIS I sholud have thought of that!

  • C8HGDT , [url=http://dlompctcbphu.com/]dlompctcbphu[/url], [link=http://kdtaicwzbyxo.com/]kdtaicwzbyxo[/link], http://ozupkfiluoqw.com/