One important behavior change in 2.12 release was WinMerge now includes all files in all unique subfolders to the compare. Previous versions just showed that there was an unique folder.
Old behavior had many problems, we really didn’t know (and hence could not show) what was contained in those unique folders. Users complained when they could not copy/delete just certain items from those unique folders. It prevented developing many new features as the result set was just part of the reality.
In fact one very reason I finally implemented this was I can implement filtering for file copying. Meaning we apply file filters when copying items. That means user can set filter to filter Subversion’s .svn -folders. And when user then copies folders WinMerge does not copy those .svn folders. This is the feature I personally miss most. And seems many users agree it would be good feature, some even submit bugs about it.
So I thought I finally got it right when I re-wrote code to have all items in all unique folders included in compare results. It was a change I had been planning to implement past 5 years. And actually I had submitted couple of patches earlier but those patches weren’t good enough to commit. So I was actually quite happy that I got it done now.
Of course there are always users that are unhappy when something changes. I was expecting that and didn’t react to some complaints other than simply explaining it is the current behavior. Until I read this message in the forum saying new feature causes user’s usual compare times to grow dramatically (from under a second to 5-10 minutes). This was quite a shocking message to read!
And what is worse, there are no good solutions to this. I’m not sure if there are even sensible solutions we could implement. No way we go back to the old buggy behavior.
The problem is not just WinMerge – it is a lot more problem of operating system and hardware. OS and hw have caches that store last used disk data to memory to prevent slow disk access. But with this kind of huge folder structures most of the data must be read from the disk. And what is worse with WinMerge, we have two different folders structures when we compare, not just one. OS and hw caches assume programs use small set of files whose data can be kept in the cache. But then comes WinMerge and compares 100 thousand of files from two folders… And caches give up.
At the moment I don’t have any realistic ideas for solutions. It might be we can optimize the unique folders case somehow. Or delay reading the data, reading data in background etc. It is sure we need to do something…
Software is sometimes so fun!
