It’s almost 2016 and no new zkanji yet. Sorry about that. Writing the program from scratch takes a lot more time than I originally thought. Also the quirks of Qt make it very very hard to work with the user interface. I think this alone doubles the development time.
There’s a positive side too of course. Most major features are now done, the last one I’m going to add is the word list printing. (There will be kanji printing too but only after the first release. First I want to make a version which can at least do what the previous ones could.)
After that I’ll start working on a more user friendly user interface. It’s mostly working around Qt, so it will take time unfortunately. I might make a test version soon if anyone’s interested, but there hadn’t been a huge interest in that in the past either. Requests if you want something like that.
To sum it all up, this means more waiting. And more work for me.
I have heard this before and read it at many places online, that generally if you can recall something 2-3 months after the last time you were exposed to it, that something is probably already in your long-term memory. Although the article on Wikipedia about long-term memory seems to give a lot of confusing contradictory definitions. Whatever, in zkanji I’ll mark a word as “learned” if you could answer it correctly after a period of 60 days. Haven’t decided about what to do with kanji yet. Probably something similar but when you could answer the written form of a word correctly that contains it. Learned kanji won’t be “unlearned” if the vocabulary goes back to the not-learned list though (after 1-2 wrong answers).
Rewriting zkanji to be cross platform requires certain changes. One of these changes is how the folder, where the program will save its user data, is decided.
Currently zkanji looks into its own data folder where it was installed, and if that’s writable (i.e. because it’s not the “Program Files” folder), it shamelessly starts dumping all its saved data there from that on. Otherwise it looks up the App Data folder’s path and uses that one. (Or at least a sub path in it.) This had the happy consequence that if you installed zkanji on an external hdd and moved it to another computer, it was automatically working like a portable installation.
I think this works fine, and I wouldn’t change it, but I don’t know the preferences of other people and operating systems. So rather than letting the program decide, the new version will ask the user where to save its data. This might be another extra click you might find inconvenient, but at least it should be safer and will only happen once.
The new algorithm will first check whether the data folder on the install path is writable at all. Because if it’s not, there’s no real question what to do. (So the App Data will be used as there is no other location it could safely write to.) If it is writable though, the program will also check for the existence of previously written user data in there. (You could of course fake the user data, making empty files with expected names. Zkanji won’t really check whether they are valid.)
Next it’ll look into the (OS dependent) application data folder, and whether user data exists there already or not. If user data is only found in one location, that location will be used and no questions asked. On the other hand if both are found (and can be written to) zkanji will assume a portable installation. I first thought that it should save the choice in its configuration (or .ini) file, but if there are two folders with user data, both will have a config file, so it would be pointless to check.
Of course you might want to override the automatic choice. In case you have a fresh install of zkanji on a portable drive, and the host computer also has a zkanji installed, the program on the portable drive will assume that it is not portable. It has found user data in the App Data location and nothing on the local install path, so there’s no real way to determine the truth.
In that case, if you want to make the install portable, there will be an option for that in the settings. Once you selected it, user data will be saved to the installed data folder, and no more misunderstandings will arise (hopefully). The other way around, that is, if you want to make the portable install to be non portable, the only way is to move the local data to the App Data location. If I’m not lazy when I get to that part, the program will do it for you. Overwriting existing data will need a confirmation of course.
Hopefully this change won’t be noticeable at all. Already existing user data won’t be deleted or moved around, just used as it is.
UPDATE: I’m rewriting the program almost from scratch in VS2013. I can’t promise anything though as I might give up halfway. If anyone’s curious about the developing code I can share it on SF. I use c++11 features and the interface is currently using Qt though I do everything to make the interface separate from the code. I use Qt containers but most of them can be replaced with simple STL. I want it to compile under Linux as well as on Windows. In theory it might work on a newer Mac but I don’t have the tools to check.
I’ll write about my experiences with Qt on the weekend or next week.
If you see this…
…don’t buy. Someone is trying to sell zkanji under a fake name. There is not much I could do about it, but you can if you tell about this to others to stop them from buying it. Thank you.
(In general anything with a true open source license can be sold without paying royalties, if it is not specifically forbidden in the license, but only if the name of the author, the product, and the way to get it for free is indicated at a clearly visible location.)
SourceForge is convenient as they give unlimited hosting space and not just for uploaded files but personal pages as well, but I wish the service was more reliable. I have uploaded the latest version of the C# code of my very simple and unsophisticated dictionary based on SQLite I wrote in C# as an exercise, and 11 hours passed since that but still nothing. Only the message “This file will be ready for download shortly”. I doubt anyone would want the sources of that not so serious “project”, but if you do, I can send it via email. (And if you were worried, I’m back to working on zkanji.)
This is a follow up of the previous post.
I owe an apology for the C# team. (See the updates.) There was an error in the test code. While deleting items randomly from my own ZSortedList, I always deleted the first item from the SortedList, which meant it had to move the whole array when freeing space at every step, making it slower. With the fixed code the performance of the two implementations are much closer to each other when working with relatively low number of items. At very low item numbers the SortedList even beats ZSortedList (not after update3), but above a given number the two lists perform similarly, but as it turns out ZSortedList is still faster.
UPDATE: SortedList never changes its capacity unless it’s told to, while I always free up unnecessary space on remove. Now that’s what I call cheating. :p
UPDATE2: My idea that having to search twice is slow was proven incorrect. Once I changed the code so my list also uses exception handling and creates a variable to insert with new, even if the position turns out to be taken, ZSortedList easily wins.
In conclusion I can say that
the built in container types (or at least the SortedList) are not as bad as I first made them look like are even worse than I thought at first, and it is possible to make better ones when working with a large amount of data where speed is critical. You can even gain minutes or hours! But plain old arrays are still much faster.
random insertion and deletion: 1024 * 200
Test1: ZSortedList first checks for a position to insert a new item, but doesn’t create it unless the position is free.
ZSortedList: 47726ms (+5959ms)
Test2: Using ZSortedList exactly like SortedList. Calling “Add” with a newly created object, and if the position is not free, handling the exception.
SortedList: 40372ms (+10371ms !!)
After changing that even the binary search is done by my own code, ZSortedList finishes the 1024*200 items test in 16000ms, which is half of the previous record, and even that could beat the built in SortedList!
After measuring both the insertion and deletion times I realized that the way I aggressively freed up data slowed things down a lot, but it’s not impossible to improve the performance simply in C# code. I’m now curious how C++ classes would measure up here but I won’t be testing that in a while.