Scrolling Lag (finally)

Let’s talk about the scrolling lag bug.  It’s the end of an era here, as I finally have a bit of closure.  Brace yourselves, this is a long one, and it doesn’t have a happy ending, I’m sorry to say.

As I’ve written about before, one of the chief difficulties in making QTTabBar work on Windows 7 was the fact the good old clunky-but-heavily-documented ListView control was replaced with the new smooth, sleek, and totally-undocumented ItemsView control.  The biggest problem is that there was no apparent way to figure out what item the mouse was over, which is important for tons of things that QTTB does.

At first the problem seemed insurmountable, but then I stumbled upon UI Automation, which seemingly solved all my problems: It allowed me to hit-test the ItemsView accurately, and even allowed me to get the bounding box of each item (something required in order to know where to position the blue arrows).  Best of all, the method was even documented!

Unfortunately, it turns out that UI Automation itself is directly responsible for the unresponsiveness that users have reported.  It turns out that every single individual query degrades Explorer’s performance slightly, but permanently.  After many queries, it becomes slower and slower and slower, and restarting the Explorer process is the only way to “reset the counter.”

Posting on various Microsoft developer forums about the problem yielded no useful results.  One forum poster suggested I contact Microsoft Support about the issue.  That sounded like a good idea, until I saw the $99 price tag.  Pay Microsoft to fix their own bug?  It think not.  However, after some more Googling, I discovered a claim that Microsoft Support will not charge you if the issue you’re reporting turns out to be an actual bug on their end.  I contacted the support line about this (yes, there’s a support line for the support line) and to my surprise, the representative confirmed this.  So, I typed up a nice little report and sent it in.

I was contacted my Microsoft’s Rob Caplan about the issue, and after some back-and-forth, he confirmed it was a bug on their end.  We looked for some workaround, but in the end, the only feasible one was “Just use the old ListView,” which I’m not happy with.  So, we went through the process of filing an official hotfix request.  That was about 2 months ago.  Finally, just a few hours ago today, Rob called me to inform me that the hotfix request had been rejected on the grounds that the “use the ListView” workaround is good enough, and the problem is apparently deep enough that a fix would too complicated.  Game over…

So, what can we do about it?  Well, as I said, the problem only affects the ItemsView, so if you use the older ListView control, you will not experience this problem.  In addition, I made several changes from Alpha 3 to Alpha 4 to minimize the number of queries made.  In particular, if you don’t want to sacrifice the ItemsView, you can disable Subdirectory Tips (or at least put them on Shift-only) and that will greatly reduce the number of queries made (which will greatly slow the speed at which the lag builds up).  Past that, all I can recommend is to use AutoHotKey to make a “restart explorer” hotkey to clear things up when it gets bad.

There is hope, however, albeit slim.  The ItemsView is controlled by an undocumented COM interface, which I have succeeded in gaining access to.  In essence, I have a whole bunch of functions with no name or parameter definitions.  One of them is likely in charge of hittesting.  If I can figure out which one it is, and how to call it, then all problems will be solved.  Of course, even assuming I’m successful, making use of an undocumented COM interface is a REALLY BAD idea in general, as Microsoft could change it without warning.  But in the absence of all other options, it looks like the only way left.  Either way, I will certainly keep the Automation code around though, so we’ll have something to fall back on.

I’m sure some will say, “Scrolling lag?  I don’t even know what you’re talking about!” or “Yeah I notice it, but it’s not that big a deal to me…”  Sometimes I envy people like that, who are able to ignore or overlook annoying problems like this one.  But me, I’m a perfectionist, and I will not feel at ease until this problem is completely eradicated.  I just hope I can do so before Windows 8 comes out and inevitably breaks everything again!

http://msdn.microsoft.com/en-us/library/ff486373%28VS.85%29.aspx

19 comments to Scrolling Lag (finally)

  • Jedijax

    Would it be totally dumb of me to say “Scrolling lag? I don’t even know what you’re talking about!”

    Either I am a very lucky man, who has never experienced the thing, although I have read about it, or I am very dull when it comes to perception.

    Could it be the issue only affects huge folders, with lots and lots of items, or it happens under certain software/hardware specifications? I consider myself a perfectionist too (very unhealthy habit), so I guess if I had experienced the bug, I definitely would have let you know!

    • I’m almost tempted to just not even tell you, because if you’re anything like me, it will bother you more once you know what it is. But, the problem is that after using Explorer for a while and letting the Automation queries build up, the responsiveness of the window will become worse and worse. Scrolling through a directory with many files is one place where the degradation is most visible. Resizing column headers in details mode is another. The good news is that it takes substantially longer for the problem to become noticeable on Alpha 4 than Alpha 3, so if you’re type that restarts your computer frequently, you’d never see it.

  • Mike Rap

    Hey bud, nice of you to post on my birthday, a real present. It is amazing that somebody goes through such great leaps and bounds to resolve bugs for free program. While i have donated to your cause, it seems as though you deserve more. Possibly you and Quizzo could and should get together for a paid model. Anyhow, as Ive said before, Alpha 4 is working great, and I have seen no issues whatsoever besides updated greatness! Hope school’s going well. Best of luck to ya!

  • Mike Connor

    You can only do your best in the given frames of reference. It’s no good beating up on yourself about problems you can’t influence. Annoying as this may be.

    Really great work up to now, and it all functions.

    I don’t mid having to restart Explorer now and again. May be of interest to those who want to do this;

    http://www.winhelponline.com/blog/how-to-terminate-and-restart-explorerexe-process-cleanly-in-windows-7-and-vista/

    http://www.mydigitallife.info/2008/10/10/tips-of-how-to-take-effect-after-installation-without-restarting-windows-xpvista/

    Regards,

    Mike Connor

  • anon

    Keeping the arrows when shift is held down sounds like a not-so-good-but-will-do workaround. Between QTTabBar, Classic Shell and 7 Taskbar Tweaker, Explorer becomes quite usable. Why did Microsoft have to screw Explorer so badly post XP? With MS’s attitude and inefficiency or inability to fix Explorer bugs in service packs, I doubt Windows 8 will be any better.

  • Norm

    Wow.
    THANKS for your work.
    Really. Many many thanks.

  • Chris James

    @Anon – Never heard of 7 Taskbar Tweaker. It’s very good! Thanks for mentioning it!

  • wickyd

    You are clearly a very talented programmer.

    It seems that it would be easier and quicker for you to create an explorer clone, where you have full control over the application than trying to hack and reverse engineer Windows Explorer.

    Surely, your time would be better spent creating a Windows Explorer clone that you can make look exactly like Windows Explorer, and let it take over the function of file management the same way as Directory Opus does?

    Why spend weeks/months/years trying to achieve something, that by your own concern could be completely broken once more with Windows 8? You would then have to start with alpha code, all over again.

    If you put the same amount of effort into creating your own explorer, you would then be able to code it the way you want with the features you want and know that you are not at the mercy of Microsoft who will make changes to their code that you have no control over.

    If I was a programmer, that is the direction I would take.

    I have searched the internet trying to find a freeware replacement for Windows Explorer, and they all suck in one way or another. A tabbed explorer, that looks as good, and has the same features is what we are looking for and that is why we support what you are doing.

    Please consider what I am saying. You could probably create a prototype tabbed Windows Explorer replacement in a matter of hours with your skill level, using the existing controls at your disposal in Visual Studio.

    Use it, don’t use it. Either way, keep up the good work.

    wickyd

  • john

    Many thanks!!

    QTTab is great and – contrary to the opinion above – I especially like the fact, that it integrates with the standard Explorer and is _not_ a replacement.

  • s802614

    hi there, i’m having a wired problem here with QTTabBar v1.5.0.0a4 in an XP system and the problem is just the QTTabBar doesnot show up in my toolbar menu. I’ve checked the three files in \assembly folder and enabled two files in the IE add-ons, and i won’t bother the “third party enable” problem cos QTTabBar works just fine in another computer with same IE configuration. Also, the QTTabBar shortcut key works well, at least i can use ctrl+alt+left/right to twist around my scrren. Now when i log in my computer with another account, the QTTabBar is just right there on the menu bar, it seems something wrong with my account. any idea of this? Great thanks!!!

  • Sartre

    Another reason to love the old ListView. Column headers in icon view, no fixed Auto Arrange, and… no scroll lag. The first two are so essential that I’ll probably always use ListView, so this is a non-issue to me.

  • xpclient

    ShiningMasamune, if they indeed don’t charge if it turns out to be their own bug, can you report the toolbar-positions-not-getting-retained-in-Windows-7 bug to them?

  • kemie

    Scrolling lag? Oh, THAT’s what was happening!

    Thank you for working so hard on getting this working :)

  • Ziktur

    Hey! Awesome work and you are still awesome!

    My question is, why do we need the blue arrows? I’ve been using OS X as my main OS and Windows as my gaming/graphic design setup. QTTB is a must have thing when dealing with lots of files and folders so I found a couple pieces of seperate software that give me specific features in OS X that QTTB offers.

    One of the features is the menus like the blue arrows provide but it is just with a simple right click and in the normal contextual menu. Could the same be done for QTTB? Instead of blue arrows, the file lists are just part of the normal contextual menu. FinderPop is the prefpane for OS X if I didn’t explain things correctly. lol

    I haven’t a clue as to if this would help or not, just my noob interpretation of what you cited since I’m no programmer. lol

  • Frank

    So no updates untill this issue is solve? This is the first site I come to everyday since August 27th(the last update)maybe I should consider changing my browsing habits. Thanks all the same.

  • Win-a-bagle

    I know you’ve constatly asked us politly about to where we should post these types of suggestions, but it’s sort of related to this subject.
    Why not add a vertical scrollbar in list view (blue arrow)? Like Windows Vista/7’s “cookie crumbs” (or “breadcrumb trails”) does in the explorer address/URL bar?

    Is there anyway to integrate that into QTTB?

  • Ralph Martin

    Have a good laugh about Windows 8 braking everything again.

    Thank you for keeping us posting. Keep up the good work.

  • Chris

    Truly glad that I have found this. Please keep up the great work!

    The title up there: Dragging Explorer into the modern era

    Kicking and Screaming from the looks of it! As a software developer on various MS platforms since 1985 they really me suicidal at times. I feel your pain and fully sympathize with you!

  • xpclient

    How or where do you report a bug without them charging you for submitting the bug? This would be very valuable to know.