View Issue Details
| ID | Project | Category | View Status | Date Submitted | Last Update |
|---|---|---|---|---|---|
| 0002715 | Dwarf Fortress | World Generation -- General | public | 2010-07-15 21:19 | 2014-09-11 21:12 |
| Reporter | TwelveBaud | Assigned To | |||
| Priority | none | Severity | crash | Reproducibility | sometimes |
| Status | new | Resolution | open | ||
| Platform | Intel Itanium | OS | Microsoft Windows XP 64-Bit | OS Version | 5.2.3790 SP2 |
| Product Version | 0.31.10 | ||||
| Summary | 0002715: Crash during huge, lengthy world gen (out-of-memory) | ||||
| Description | When doing world generation of large (257x257), ancient (10,000 years) worlds, Dwarf Fortress crashes with an Out Of Memory error, which may show up as a hang and an empty dialog box -- the game is so out of memory Visual C++ can't even crash properly. In my tests, this has happened at around 1.92GB of memory allocated, or around Year 2050, with about 7.5 million events. | ||||
| Steps To Reproduce | 1. Launch Dwarf Fortress. 2. Design New World With Parameters 3. Press Esc to acknowledge the alpha status. 4. Make a copy of the LARGE ISLAND paramset (257x257), and edit this copy to include an End Year of 10000. 5. Press Enter to begin world generation. 6. ... 7. PROF- wait... CRASH! | ||||
| Additional Information | Making a 64-bit build of Dwarf Fortress would probably be Very Very Fun, but as long as you're not mixing pointers and status codes (0x7FFFFFFF and below as a thingie, 0x80000000 and above as an error), one way to stave off the problem would be to use /LARGEADDRESSAWARE when compiling. This will allow Dwarf Fortress access to double the normal amount of memory when running on a 64-bit computer, even though it's still 32-bit. With luck, I might make it to the year 3050! :P | ||||
| Tags | No tags attached. | ||||
|
|
My second test, this time with the Large Address Aware setting hacked into the executable, appears to be going well. After 6063 years and 7.2 million events, Dwarf Fortress went over the 2GB limit and kept going. It looks like legends are what's driving memory usage, and that using /LARGEADDRESSAWARE should be safe. Although I'm running on an extremely weird configuration of Windows XP 64-bit for Itanium Systems, this issue is also present on Windows XP 64-bit for Extended Systems, and the 64-bit editions of Windows Vista and Windows 7. (The issue also exists on the 32-bit editions of all versions of Windows, but there's not much to do that can fix it there.) While Windows may have plenty of free memory to give out, after a while it has no place in the running Dwarf Fortress to put it. In 32-bit versions of Windows, the processor can keep track of 4GB of "memory space" at a time, 2GB reserved for the program, and 2GB reserved for Windows. Thus, most programs could only access 2GB of memory each. Several developers took advantage of the existance of the reserved space, and, if there was an error making an object, would use the error code in place of the object pointer, which worked because conveniently error codes exactly overlapped the reserved space. In 64-bit Windows, when running a 32-bit program, Windows is located entirely outside of the 4GB "memory space" -- since it has 256TB of memory space to play with -- but the program is still limited to 2GB to avoid breaking programs that use that technique. However, adding a special compiler flag tells Windows that the technique is not in use, which means the program can safely handle up to 4GB of memory space. This is still less than a native 64-bit program could access, but double what would normally be available. This is realllly really easy. Another option is to use "memory-mapped 'files'". This technique can be used to allocate much more than 2 or 4GB of space, and provides the program with a smaller, movable "view" of the space. Thus, while you can't access the whole thing at once, you can "scroll through" as much memory as you want, limited only to how much at a time in that small window. Raymond Chen has a brief article summarizing the technique: http://blogs.msdn.com/b/oldnewthing/archive/2004/08/10/211890.aspx This is a little harder, since it requires a rewrite of how to access whatever you're accessing, but offers the greatest compatibility for the effort required. A final option is to compile a special 64-bit edition of Dwarf Fortress, which doubles the size of each and every pointer. This can only be done after making sure that pointers are never stuffed into longs/DWORDs, and longs/DWORDS are never converted into pointers, since it requires a long long or a QWORD to fully represent the size of a pointer in 64-bit code. It also will not run on 32-bit editions of Windows. However, this will give it room for up to sixteen terabytes of memory, more than can possibly be installed or swapped out. This... amounts to a complete rewrite. And probably will break save compatibility. Nevertheless, it is a possible option. Anyway, I've been letting the altered Dwarf Fortress chug away as I've been typing this, and it looks like "Nitom Anur", the Planet of Wonder, has 1,394,967 historical figures in 12,152,449 events in 10,000 years, still somehow in the Age of Myth, with a memory high water mark of 3,359 MB, and no problems whatsoever. ☺ |
|
|
This seems more like a suggestion then anything else. The only thing Toady could do with this report bugwise would be to either force you not to set large history dates (I believe culling history is there to allow this) or put in better error handling to catch the program going over memory limit. |
|
|
Cryten: If the program is crashing, that's a bug, no two ways about it. Disallowing extremely long histories, better error-handling, or some other method of dealing with this would certainly be an improvement over the program running out of address space and crashing. |
| Date Modified | Username | Field | Change |
|---|---|---|---|
| 2010-07-15 21:19 | TwelveBaud | New Issue | |
| 2010-07-15 23:51 | TwelveBaud | Note Added: 0010409 | |
| 2010-07-16 04:08 | Logical2u | Relationship added | child of 0000136 |
| 2010-07-16 06:16 | Cryten | Note Added: 0010415 | |
| 2011-07-08 14:37 | G-Flex | Note Added: 0018156 | |
| 2012-02-21 06:51 |
|
Relationship replaced | related to 0000136 |
| 2015-12-06 18:51 |
|
Relationship added | related to 0009243 |
