Frhed 1.3.10 Beta

Couple of days ago I did the Frhed 1.3.10 Beta release. This beta release is the last of 1.3.x releases before the 1.4.o stable release. Expect the stable release in about two weeks.

The start of the Frhed development has been very very interesting. The codebase we started working was good, but old. There weren’t many bugs, but the code, GUI etc was designed for Windows 9x. So the most important thing so far has been to make sure Frhed works correctly in latest Windows versions. And update the GUI to match current practices and guidelines. We’ve advanced a lot, but a lot remains to be done after 1.4. release too.

Download counts of Frhed alpha releases have positively surprised me. I expected to get few dozens of downloads. After all it was an alpha release. Intended for testing, not for production use. But there are already over 4700 downloads. So people have already found the tool. It will be interesting to see how much downloads the stable release gets later.

Frhed has been also very refreshing project for me personally. As it is in very early stages (from my point of view we just have started with it). We can just rewrote or remove things we don’t like. No need to think about hundreds of thousands of users. It is sometimes quite hard to improve things in WinMerge since bigger changes always affect to users. And not always in positive way. With Frhed we just don’t care yet.

For Frhed we start really caring perhaps after couple of stable releases. When we have some idea what users want and need from the software. Do they want simple and fast hex editor? Or do they want more features rather than simplicity and speed? We’ll find out but it takes some time. Then we must choose our own way to go.

Frhed 1.3.4

Today I made the Frhed 1.3.4 alpha release.  As I’ve outlined in my forum post, I expect to have Frhed 1.4 stable release in about month.

For that target this 1.3.4 alpha is the most important release. It already is pretty close to stable release in features. Maybe we’ll add some new features, but nothing major. Most changes will be smaller fixes and improvements. Manual is one part I’d like to improve but lets see if I find time…

Previous Frhed releases were more of setting the infrastructure (dowloads, web page, trackers, etc etc). Frhed 1.3.3 already had lots of things in place but missed many required bug fixes and improvements. So in my book Frhed 1.3.4 is also the first real Frhed release. And I like it!

Please test and use it. If you find bugs, please report them to our bug tracker. If you have ideas for improvements, submit them to our feature request tracker.

Case collision in Frhed repository

I just hate that Windows doesn’t care about case in file/folder names. Today I hate it very very much. I’ve spent hours and hours trying to get working Mercurial repository converted from Subversion repository.

It should be trivial thing to do. I’ve converted many other repositories without problems. And indeed I’ve been happily using the import I did couple of months ago. Then at Monday when I was traveling without network access I decided to do something I hadn’t do before with this repository: I wanted to do a new clone and develop with it. I expected it just works as Mercurial usually does. BUT cloning aborted with error message “abort: case-folding collision between FRHED/frhed.dsp and FRHED/FRHED.dsp“. WHAT??

Digging into history I noticed there is commit by Jochen (SVN rev 46) which renames FRHED/FRHED.dsp to FRHED/frhed.dsp. That looks so innocent change and indeed should not cause problems. But for some reason I haven’t yet figured out the FRHED.dsp is not really removed from the converted Mercurial repository. So now when I try to update the working copy (clone, update etc) there are two similar filenames (as Windows sees them).

I expect this is two-fold problem. First the Subversion doesn’t do real renames for the files, but it does copy + delete for the file. And then (I suspect) hgsvn doesn’t detect the rename correctly as the filename doesn’t change (from Windows point of view).

There is a page in Mercurial wiki about fixing case collisions. I’ve tried many variations of that. It basically works as it creates repository I can clone. But as the “fixing” commit is naturally the last commit in the repository it doesn’t help with the problem unable to update working copy to earlier revision. I’m afraid I have to live with this restriction.

Out of interest I’m also trying Mercurial Queues to edit the history and remove the FRHED.dsp file after the rename. In hope I can get a fully working repository.

Yes, I know. Editing the version control history is bad idea. You simply should not do it. Especially with distributed version control system like Mercurial. Mercurial wiki has a good page about this.

But I want to experiment this (I’m sure I learn a lot about Mercurial in the course). So far it seems it can be done, but it requires hand-editing many revisions. And indeed I just realized I tried to do it wrong way around.. Whoops.

The principle here is MQ exports the history as set of patches. Then the patches can be edited, reordered, removed etc. And applied back to the repository after the changes. My idea is to remove the commit that renamed the file (so the original FRHED.dsp filename remains untouched). And then edit further patches so that they edit FRHED.dsp file instead of frhed.dsp.

As I said above I already started doing this wrong way – I removed the rename commit and started editing patches so that they edit the frhed.dsp file instead of FRHED.dsp. But the problem is of course that if I remove the commit, there never will be frhed.dsp file… Doh.

After some more experiments I’m stuck with new problem. Edited patches don’t apply cleanly and at the moment I’m still wondering why. I mean, the patches don’t apply for files whose diffs I don’t edit in patch files.  If I just remove hunks for editing FRHED.dsp why hunks for heksedit.dsp fail… This is a bit weird. But I’m new with this MQ stuff so perhaps I just did something wrong. I need to learn more about MQ.

But I’ll leave that for tomorrow. I’m still positive I can get working repository out of this.

Update: After several attempts I finally managed to fix the Mercurial repository.

Things got wrong in SVN rev 46 (as mentioned above). The hgsvn conversion script records rename of FRHED/FRHED.dsp to FRHED/frhed.dsp as copying of FRHED/FRHED.dsp to FRHED/frhed.dsp. So it won’t remove original file, just creates a new copy with new name. Which only differs by case and causes Windows to not see it as two different files.

And then things go totally wrong in conversion. The script generated diffs that made same changes to both FRHED/FRHED.dsp and FRHED/frhed.dsp files. Which in addition to all other problems also caused the patches to fail to apply by hand (as the first change already applied the change and second was then invalid).

Anyway, the working solution I found was to convert the faulty revision to real rename. I did this with MQ (see above). I reverted the revision 46 (SVN) and replaced it with real (Mercurial) rename of FRHED/FRHED.dsp to FRHED/frhed.dsp. Next problem was there were set of patches referring to now inexisting file. Simple solution was to do a search in files for FRHED.dsp for the patch files and remove all changes to that file. Once I had fixed the patch set I simply re-applied it to repository.

And the end result is the FRHED.dsp file disappeared. And I have exactly the repository I wanted.

I’m not sure how to avoid this kind of problem in the future. Best would be to avoid this kind of renames. But if somebody does the rename then there is no way to really backout it from Subversion repository. So sometimes manual fixing is needed. Luckily I now know how to do it in few minutes.

The exact procedure is:

1. convert the SVN repository to Mercurial using hgsvn
2. CD to Mercurial repository
3. initialize MQ: hg qinit
4. export history to patches, including the faulty revision: hg qimport -r FAULTY:tip
5. apply the faulty patch: hg qpop - copies FRHED/FRHED.dsp as FRHED/frhed.dsp, but nothing happens in the filesystem
6. fix the result: hg mv FRHED/FRHED.dsp FRHED/temp.dsp; hg mv FRHED/temp.dsp FRHED/frhed.dsp – this results in Mercurial seeing real rename. We must use temp as filename as intermediate step.
7. tell MQ that the patch was modified: hg qrefresh
8. edit patch set in .hg/patches folder and remove references from all patches to FRHED/FRHED.dsp file
9. apply the fixed patch set: hg qpush -a
10. delete the patch queue: hg qdelete -r qbase:qtip
11. Now we have fixed Mercurial repository

New Frhed 1.3.2 alpha release

Yesterday I made second release in two days. Luckily Frhed releases are still a lot less “major” releases than WinMerge stable releases are.

Frhed 1.3.2 alpha release is still setting the basics for the project. There is no user-visible changes (other than couple of new separators in File-menu) compared to earlier releases. But this releases installs important documents like license text and contributors list with other documentation. Also for now on I’ll be using Visual Studio 2003 instead of VC6 for compiling. Hence the runtime files for VS2003 were added to install.

Switching the compiler is not always easy. But For Frhed it was pretty painless. And I have quite a lot of experience from VC6 to VS2003 update from WinMerge project…

There is a lot of code cleanups which is kind of base job for further improvements. I also finally got the release script working so further releases are easier and faster to do.

Frhed 1.3.1 alpha is available

Few hours ago I uploaded Frhed 1.3.1 alpha release to the Sourceforge.net. This is the first Frhed release from me and new Sourceforge.net project.

Although I believe this is pretty stable and solid release, this is still meant being a snapshot of our current status. And give users something they can try and test. So what has happened after I took over the original Frhed project by Raihan Kibria?

I’ve re-created the installer with the latest NSIS version. I personally don’t like NSIS much but didn’t want to totally change the installer either. Maybe later.

Jochen has been doing a lots of refactoring and fixing. Lots of that is because of converting the editor part as an embeddable component (for WinMerge to use).

Tim submitted a new icon for the program. The old icon wasn’t so bad,  but it was just too old-style from W98 days. We needed something more modern and colorful.

Windows XP theming is now supported (the common controls manifest is embedded). So the application receives more modern-looking common controls in the GUI. The new icon and the new look are the really visible changes.

As this release shows we are really improving this software. It wasn’t just that I wanted to create a new Sf.net project. And we are improving it as a stand-alone software, not just as WinMerge component! I personally feel that Frhed as a stand-alone program is a lot more important than as embedded to WinMerge. But as a stand-alone software it probably gets few thousands of downloads. And as a WinMerge component it gets millions of downloads.

Unfortunately I haven’t had a time to really plan Frhed’s future plans yet. Basically I’m still figuring out things like manual’s format, building of the software etc practical things. Once I get the basic stuff sorted out I have to start thinking about Frhed as a software. And as a WinMerge component.

Introducing Frhed

Oh my, my second blog post today. This is the long overdue introduction of the project called Frhed.

One weakness of WinMerge for some users has been poor binary file handling. WinMerge can compare them (tell if the files are identical or different) but not much else. Also binary file detecting has been quite strict and some perfectly good looking text files have been determined being binary files.

To overcome these restrictions Jochen implemented binary file viewing for 2.8.0 stable release. It encodes binary files to text files for textual compare. And shows results like normal text file compare. But bytes that cannot be presented as characters are shown as their byte value. I liked this approach and in my personal opinion it was all we needed. Real binary file compare would require rewriting lots of WinMerge code.

Few months ago Jochen submitted a patch that added new binary file compare + editor code to WinMerge. Jochen basically integrated existing Open Source binary file editor called Frhed to the WinMerge. Doing lots ot refactoring in the process. One big refactoring is to move most of frhed code to dll we can load to WinMerge and show the editor window. This dll editor component is called heksedit.

There was one real (and for me a big) problem with this. The Frhed upstream development weren’t been active in years. Who would fix the bugs in Frhed? It could not be only the WinMerge developers. We have real problems to fix bugs in WinMerge to take yet another codebase to our maintenance. Look what happened with CrystalEditor. WinMerge is perhaps the only project publicly developing it.

I still thought we had a real use for the Frhed and it looked like a nice stand alone editor too. So I contacted the original Frhed author and asked if he would allow me to take over the project maintenance and create project for Frhed into Sourceforge.net. As the original author agreed I can do this I then created the project.

Goals of the Frhed project are:

• to develop stand-alone Frhed binary file editor.
• to develop heksedit dll as an embeddable editor part of the program. Currently WinMerge is the only client but I don’t mind other clients.

So far my personal target for Frhed has been towards the first alpha release. Which means getting basic things in order, like building, manual, versioning etc. I think there are only couple of details left and I can create the first alpha release next week. The release has been delayed mostly by lack of time (WinMerge release and all kinds of other stuff).

One very important thing to note is that WinMerge project will not develop Frhed or heksedit other than perhaps some WinMerge-specific changes (should be very minimal). All Frhed-related patches, bugs etc must be submitted to the Frhed project. New translations and translation updates must be submitted to the Frhed project, not to the WinMerge project. WinMerge project only gets certain revision or release of Frhed as its binary file editor.

Somebody might wonder if all this trouble is really worth of it. I already gave one example why it is – the CrystalEditor. The CrystalEditor was developed years ago. I think it was somewhat active when Dean (?) decided to use it in WinMerge. But since then the project has been practically vanished from the net and WinMerge is only remarkable project (that I know of) using it. There was project called Textcenter that tried to develop CrystalEditor further. But even that project has been inactive for years. For us WinMerge developers it would be ideal we could just pick release of some editor component and trust it just works for us. And we could concentrate on developing features that are related to comparing and merging, not basic editor stuff. We do that with other libraries like PCRE and expat. We don’t fix/develop them.

In the lucky case the Frhed project itself gets some interest by people and we’ll get new people helping in developing it. Please remember Frhed is a separate project – developing Frhed does not mean you need to touch WinMerge code. And Frhed project has no limitations WinMerge has – it does not use MFC so it can be developed using free version of Visual Studio.