View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0000332 | Dwarf Fortress | Dwarf Mode -- Stockpiles | public | 2010-04-03 17:27 | 2010-07-26 07:17 |
| Reporter | oliver | Assigned To | Toady One | ||
| Priority | normal | Severity | minor | Reproducibility | have not tried |
| Status | resolved | Resolution | fixed | ||
| Product Version | 0.31.01 | ||||
| Fixed in Version | 0.31.09 | ||||
| Summary | 0000332: Raw files parsed in strict alphabetic order in Linux but not Windows, causing ID mismatches for saves moved between systems | ||||
| Description | Usually, on embark, bituminous coal and lignite are considered to be an "Economic" stone, in terms of the stone stockpile categorization. However, my current fort (about 3 years in) has suddenly decided that they should be in the "Other" category. The same fort was OK previously. They are still red ("don't use as misc stone") in the 'z'->Stone screen. This is possibly related to 0000219 as I seem to have lost the ability to make coke from those two stone types too. | ||||
| Tags | Save Included, SDL-only | ||||
| has duplicate | 0000219 | resolved | Smelting reaction ingredients/products change during gameplay, possibly Wine-specific |
|
|
I can confirm this. It has happened to me in two different forts now. Bituminous coal and lignite are red in the stones screen by default, but they don't list the "make coke" ability, and the smelters do not get the "make coke" menu option, even though loads of the resource are available. |
|
|
check (z) stone and see if it has changed to green (which it can do if you find a lot). I'm wondering if this also moves it in the stockpile now as well. |
|
|
oliver and Zet: what OS are you using? |
|
|
In my case this was the same save that was transferred from XP to Linux/Wine and suffered from the disappearing reaction problem as described in 0000219, so it's quite likely related. |
|
|
Cross-posting this from 0000219, as I've been told that it's an occurence of this bug. OS: Windows XP Professional 32Bit I've had the same occurence with Bituminous Coal and Lignite: - They're still red in the z stone screen. - They don't list the 'make coke' ability. - They are in 'ordinary stone' in the stockpile menu. - It's possible that making coke still worked for some time as I only noticed this when my repeating orders got cancelled. - Now I can add the jobs via manager, but it states 'Urist McSmelter cancels Make Coke from Lignite: needs rock' and cancels. - I got a few reoccurring crashes afterwards that seemed to stop when I cancelled the manager coke orders, so ordering coke might eventually lead to crashes. I'm not positive on this one though. - I can still make Pig Iron and Steel from charcoal coke. - Saving and reloading did not help, smelter is not in a burrow, save was never transferred. |
|
|
CptFastbreak: do you have anything interesting in your errorlog? Also, it might be helpful to upload that save to http://dffd.wimbli.com/ |
|
|
Here's the save file: http://dffd.wimbli.com/file.php?id=2261 The errorlog doesn't contain anything related to the smelter or to coke, just a bunch of "path fail" and "loop path fail" mostly related to the construction of buildings and the fleeing from invaders. |
|
|
I encountered this bug after transfering a savegame made with 31.04 - legacy version to 31.05 - sdl-version (tried linux as well as win). Using the save with the 31.05 - legacy version works fine. I also created a new world with the 31.05 sdl - version (win) and here the reactions listed in the z-stone menu are correct, although most of the ores now show the ability to make sharp blades while Obsidian isn't listed at all. |
|
|
I also encounter this bug. I used wine before and loaded my save to linux and now I can't make coal. Interestingly, if I save and load into wine (new and old windows versions) the save game works fine. When I go back to linux and it is broken again. So i guess there is a linux-only bug here. |
|
|
So one way to work a round this is to use links "ln -s path_to_linux_df/data/save/ path_to_win_df/data/save" then just open with wine queue up coal creation then exit open in linux and the coal creation will start up. Not a great solution but works for now. |
|
|
Actually, not only the coke creation tasks are missing from the smelters. All make <alloy> from <bars/ore> tasks are missing, too. This happens when trying to use Wine generated world on native Linux DF and vice versa. |
|
|
I have managed to work out how to reproduce this. Someone may want to reopen 0000219 with this info, since it's more directly relevant there. As suggested in a comment on 0000219, it is to do with the order in which the raws are loaded. I reproduced this on Linux by the following steps: * Create a new region, embark * Verify that bituminous coal has the right reactions. * Save the region. * Run 'ls -lU | grep inorganic' in data/save/regionN/raw/objects. -U provides an "unsorted" list of files that will reflect the order that the OS-level directory-scanning APIs return. In my case I get this order: inorganic_other.txt inorganic_stone_layer.txt inorganic_stone_mineral.txt inorganic_stone_soil.txt inorganic_metal.txt inorganic_stone_gem.txt * Do something to inorganic_stone_mineral.txt (which contains the bituminous coal definition) so it is in a different position in the order. It will be very OS and filesystem specific what you need to do here. In my case, on my Linux install, renaming it to "inorganic_test.txt" worked, producing this order: inorganic_other.txt inorganic_stone_layer.txt inorganic_stone_soil.txt inorganic_test.txt inorganic_metal.txt inorganic_stone_gem.txt * Load the saved region, check reactions, note that bituminous coal is missing reactions and is considered an "other" stone in the stockpile lists. * Save * rename the raw file back to its original position * Reload * The reactions are fixed. I'm guessing there is an ID stored internally somewhere in the region save that is just an index into the stone types loaded from the raws, and that index can end up pointing somewhere different depending on the order the raws are loaded. So moving between Wine/Linux/Windows etc, or just bad luck with what the filesystem does under the covers, has a chance of changing the ordering of the raws and causing chaos. Fun times! Possible fix: sort the raw filenames before loading them, since that should be a stable order regardless of what the filesystem does. But that will break old saves :( (Might be worth alerting Toady to this one, since it has the potential to cause all sorts of savegame problems - backups that don't work, can't transfer saves between different systems, savegames mysteriously going bad for no apparent reason, etc) |
|
|
I'm having this bug as well. I started the game under Wine. Later on I switched to Windows, mainly because the game ran so much faster under Windows. I moved the game files to Windows partition, and also upgraded the game a few times. I think I started with 31.03, running 31.08 now. Then I noticed I can't make coke from bituminous coal. I can use the manager noble to add the job, but the dwarfs just cancel it, saying "..cancels Make coke from bituminous coal: Needs melanite." I tried running the game under Wine again. I first copied the game files back to the Linux partition. Now the smelter shows "Make coke from bituminous coal" and "Make coke from lignite" options, which were not available under Windows. However, as the game files have been moved from one partition to another and then copied back again, I'm not sure what the original order was when I started the game. The Linux partition is in ext3 format, and Windows partition is in NTFS format. This is the order inorganic* files are now listed in the Linux partition: inorganic_stone_mineral.txt inorganic_stone_layer.txt inorganic_other.txt inorganic_stone_soil.txt inorganic_metal.txt inorganic_stone_gem.txt Now if I run the native Linux version with the exact same save files, linked from the Wine directory so they should be listed in the same order, I'm again unable to create coke. Also if I try running the game straight from the Windows partition under Wine, I'm unable to create coke again. The inorganic* files are listed alphabetically in that partition. In short: the only way to create coke now is to run the game under Wine from the Linux partition. |
|
|
If I rename those files, for example inorganic_stone_mineral.txt to inorganic_00_stone_mineral.txt (so it comes first in alphabetic order) I can then create coke in Windows too. But, looking at the stone data in the z-stone-menu reveals that everything else is borked too. I could for example make bismuth bronze bars from cinnabar or brimstone. Everything in that screen changes depending on the order I rename those files. So there's like ..umm.. 720 (?) different combinations how the inorganic* files can be ordered? I can't find the correct order, so I guess I'm better off starting a new game anyway. |
|
|
oliver -- does that mean 0000219 can be reproduced without using Wine? If so, I'll reopen it. Thanks for investigating, by the way. |
|
|
Yes, I believe that both this bug and 0000219 have the same cause, a change in the load order of raws. That seems to be most commonly triggered by moving saves to/from Wine, but it could trigger in other ways too. |
|
|
Having dug into libgraphics a bit: - POSIX systems (Linux/OSX) use glob(), which sorts results - Win32 uses FindFirstFile()/FindNextFile(), which does not sort results |
|
|
I'm having strange reaction results without changing the OS. Linux only, no wine, but instead of bituminous coal magnetite is used and lignite reaction has vanished. Haven't changed the raw's either. Some days ago everything worked fine. Chromite now has the "Make coke from Lignite" tag. Could a kernel update change the sort results and produce this behavior? That's the only major system-change that happened in the last couple of days. |
|
|
I've been running in Windows from a shared drive on a Linux server. When I moved it to a shared drive on a Windows server, it seems to have caused this same problem. So it seems it still can be caused just by the servers having a different OS, even though I've been executing DF from the same workstation. |
|
|
There's a patch to libgraphics that makes the sorting of filenames explicit (and case-insensitive) at http://s3test.randomly.org/df/332/dflibgraphics-file-sorting.txt Only tested on Linux. I can't build/test on Win32 or OSX. |
|
|
Reminder sent to: Baughn Hey Baughn -- sounds like this is a fix you can add yourself. oliver posted a patch at 0000332:0009551 |
|
|
Toady says this shouldn't matter, but that something is obviously busted. It'll get fixed at a deeper level than just sorting the files. |
|
|
Yeah, I can get this to reproduce strictly within one version on Windows by changing the order of the files. However, the raw loader reports that the game is sorting them in with the proper, original order (the regions store the tokens of the individual objects, so file order should never matter). It seems there's some auxiliary information somewhere that's reacting badly to how the files are set up. My guess is that it is loading in the files, pre-calculating some reaction information, and then only doing the token sort afterward, but I have to check the order. If that's what it is, I just need to delay the pre-calc and everything should be fine all around. I can hope to preserve save compatibility (and get it working where it wasn't before), but no guarantees of course. |
|
|
Going to mark this one of for 0.31.09. I fixed the erroneous pre-calcs I found, they would cause it, and I can no longer reproduce it after the fix. Hopefully it'll be good now. |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2010-04-03 17:27 | oliver | New Issue | |
| 2010-04-26 07:22 | Zet | Note Added: 0005113 | |
| 2010-04-26 07:25 | DoctorZuber | Note Added: 0005114 | |
| 2010-04-26 10:31 |
|
Note Added: 0005138 | |
| 2010-04-26 10:46 |
|
Relationship added | child of 0000219 |
| 2010-04-26 15:52 | oliver | Note Added: 0005196 | |
| 2010-04-28 09:16 | CptFastbreak | Note Added: 0005539 | |
| 2010-04-28 09:16 | CptFastbreak | Note Edited: 0005539 | |
| 2010-04-28 12:57 |
|
Note Added: 0005568 | |
| 2010-04-28 16:13 | CptFastbreak | Note Added: 0005603 | |
| 2010-05-07 10:33 |
|
Relationship replaced | related to 0000219 |
| 2010-05-16 23:52 |
|
Summary | Coal / lignite mysteriously move to "other" stone type => Bituminous coal / lignite mysteriously move to "other" stone type |
| 2010-05-16 23:54 |
|
Tag Attached: Save Included | |
| 2010-06-05 03:07 | Eskarion | Note Added: 0007711 | |
| 2010-06-13 01:49 | Phancy Physicist | Note Added: 0008257 | |
| 2010-06-13 23:59 | Phancy Physicist | Note Added: 0008391 | |
| 2010-06-14 12:14 | jeti0 | Note Added: 0008420 | |
| 2010-06-28 01:49 | oliver | Note Added: 0009206 | |
| 2010-06-28 01:51 | oliver | Note Edited: 0009206 | |
| 2010-06-28 01:55 | oliver | Note Edited: 0009206 | |
| 2010-06-29 14:53 | uggi | Note Added: 0009325 | |
| 2010-07-01 07:55 | uggi | Note Added: 0009456 | |
| 2010-07-01 08:07 |
|
Note Added: 0009458 | |
| 2010-07-01 08:08 |
|
Summary | Bituminous coal / lignite mysteriously move to "other" stone type => Bituminous coal / lignite mysteriously move to "other" stone type, due to raw files not being parsed in strict alphabetic order |
| 2010-07-01 18:14 | oliver | Note Added: 0009484 | |
| 2010-07-01 20:35 |
|
Relationship replaced | has duplicate 0000219 |
| 2010-07-01 20:36 |
|
Summary | Bituminous coal / lignite mysteriously move to "other" stone type, due to raw files not being parsed in strict alphabetic order => Raw files not parsed in strict alphabetic order in Linux, causing ID mismatches for saves moved between systems |
| 2010-07-01 20:36 |
|
Sticky Issue | No => Yes |
| 2010-07-02 05:05 | oliver | Note Added: 0009509 | |
| 2010-07-02 05:24 |
|
Summary | Raw files not parsed in strict alphabetic order in Linux, causing ID mismatches for saves moved between systems => Raw files parsed in strict alphabetic order in Linux but not Windows, causing ID mismatches for saves moved between systems |
| 2010-07-02 05:52 | m4st0r | Note Added: 0009511 | |
| 2010-07-02 06:52 | m4st0r | Note Edited: 0009511 | |
| 2010-07-02 09:43 | numBone | Note Added: 0009527 | |
| 2010-07-02 19:24 | oliver | Note Added: 0009551 | |
| 2010-07-02 19:45 |
|
Tag Attached: SDL-only | |
| 2010-07-02 19:47 |
|
Note Added: 0009554 | |
| 2010-07-03 05:03 | Baughn | Note Added: 0009568 | |
| 2010-07-03 05:21 | Toady One | Note Added: 0009571 | |
| 2010-07-03 05:21 | Toady One | Assigned To | => Toady One |
| 2010-07-03 05:21 | Toady One | Status | new => acknowledged |
| 2010-07-08 03:59 | Toady One | Note Added: 0009774 | |
| 2010-07-08 03:59 | Toady One | Status | acknowledged => resolved |
| 2010-07-08 03:59 | Toady One | Fixed in Version | => 0.31.09 |
| 2010-07-08 03:59 | Toady One | Resolution | open => fixed |
| 2010-07-12 12:53 |
|
Sticky Issue | Yes => No |
