View Issue Details

Jump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
0003942Dwarf FortressDwarf Mode -- Economicspublic2011-02-10 15:532013-06-07 18:02
ReporterTer13 Assigned ToToady One  
PriorityurgentSeveritymajorReproducibilityalways
Status resolvedResolutionfixed 
Product Version0.31.18 
Fixed in Version0.34.06 
Summary0003942: Dwarves owning broken clothing. Clothes don't rot, ANNIHILATES FPS.
DescriptionThere is something seriously wrong with the way Dwarves handle clothing ownership. My FPS, and the FPS of a friend have suffered continually. If our fortress has 200 dwarves, we manage 100 FPS, but if those dwarves wear clothing (we turned it off and tried it), the FPS steadily decreases as more and more dwarves wear out their clothing.

It seems that Dwarves are constantly checking to see if they need new clothes once they own a broken piece of clothing, and this is causing excess processing to be done each frame.
Steps To ReproduceStart fortress with dwarves with clothing. increase population to 200 before clothes break. Good FPS.

Wait until clothes break. Shit FPS.


Start another fortress with the dwarven civ not wearing clothes. Good FPS no matter what.
Additional InformationTurn off the wear clothing behavior in the civ. This will clearly show that there is something very, very wrong with item ownership.
TagsNo tags attached.

Relationships

related to 0002481 resolvedToady One Dwarves will claim new clothes when theirs is worn (xClothesx) but not wear it 
related to 0004207 resolvedToady One Inescapable, inevitable, intolerable lag 

Activities

user6

2011-02-11 08:38

  ~0015082

It would be helpful to post saves that demonstrate this behavior. http://dffd.wimbli.com/

Overall object count is suspected of causing the biggest FPS hit. Are you able to exclude this factor? I.e., when you have have two fortresses, one with clothes and one without, are the item counts still on par?

Ter13

2011-02-11 09:46

reporter   ~0015086

Last edited: 2011-02-11 10:05

 The idea that item count affects FPS is indicative of overall design issues. Items shouldn't be processed every frame unless there is behavior connected to those items.

The item counts should have been roughly similar, as without clothes, I was able to carry on much longer, and as such produce a much larger quantity of items, all the while my FPS remained stable.

I am very certain that there is a bit of code running unintentionally due to the owned and broken clothing. I believe the dwarf is constantly seeking the nearest replacement for that broken clothing, even though he has a second bit of clothing already. I believe that this is a leading cause to FPS hits, and should be pretty easy to profile and confirm.

Kogut

2011-02-14 15:10

reporter   ~0015104

How can I disable clothes?

IT 000

2011-02-14 15:40

reporter   ~0015105

remove [CLOTHING] and [SUBTERRANEAN_CLOTHING] from the dwarven entity file.

Kogut

2011-02-15 02:34

reporter   ~0015107

I did this, but on embark dwarfs are still wearing sth. Is it supposed to happen?

hyndis

2011-02-16 11:17

reporter   ~0015122

I just delete clothing from both the entity files and also in the raws. I leave the armor but clothes are deleted outright. Works fine.

user6

2011-03-16 18:19

  ~0016309

Reminder sent to: Ter13

We need a save for this report. Please upload one to http://dffd.wimbli.com/

AbuDhabi

2011-03-17 01:32

reporter   ~0016318

Last edited: 2011-03-17 13:36

 Do you think that modifying clothes to have the [HARD] token solve this problem? This would prevent them from rotting at all, like armour does.

EDIT: It seems that just modding out [SOFT] isn't sufficient to battle wear and tear. Though much slower, the clothes do get used up.

Orkel2

2011-03-30 14:21

reporter   ~0016816

Last edited: 2011-03-30 14:21

 This is happening to me too.

Good clothes, ~ 100 FPS on 120 dwarves

Clothes starting to rot, FPS has steadily dropped to 25 in the past year or so.

user6

2011-03-30 14:39

  ~0016818

Reminder sent to: Orkel2

Please upload a save demonstrating lag from rotting clothes. Bonus points if you can also provide a save from before the clothes start rotting. http://dffd.wimbli.com/

profit

2011-03-31 00:16

reporter   ~0016827

I too would really like to see this... I would like to test and see if removing their clothing via putting them into the military with blank uniforms that replace clothing would fix it as well.

profit

2011-03-31 00:39

reporter   ~0016830

I just tested this on one of my forts, using the military feature I was able to get everyone out of their clothes... (In life I usually just have to ask and they drop) Many of these clothes were completely deteriorated, but I did not experience a FPS gain after I returned the dorfs to civ state even though they did not put on new clothing. ( I forbade it) I am unsure where the FPS boost is coming from but I have a sneaking suspicion it has something to do with the rot code or contaminants not the dorfs trying to replace them.

InsanityPrelude

2011-03-31 00:49

reporter   ~0016831

Does anything happen if you destroy the rotted clothes? (e.g. throw into magma)

AbuDhabi

2011-03-31 01:03

reporter   ~0016832

Destroying them is difficult, because they are owned, and therefore can't be dumped.

Kogut

2011-03-31 08:01

reporter   ~0016850

child of 0002481?

InsanityPrelude

2011-03-31 09:21

reporter   ~0016858

Oh, right. I forgot about that.

Dekon

2011-04-09 08:40

reporter   ~0017232

Seems common to me as well. My FPS plummets after any invasion attempt until I can manage to clean up all their crap and put it through an atom smasher. Which means my dwarves end up spending a ludicrous amount of each month just playing clean up from the last invasion.

alcohol_dependent

2011-04-09 17:39

reporter   ~0017259

Last edited: 2011-04-10 10:31

 I am going to try removing clothes as well and will post my results.. I have a 3 ghz processor, typically get a big slowdown around 80 dwarves. Never go much beyond there but if I leave temp on with a pop cap at 80 my FPS will slow to 50 with a moderately inefficient fort (35-40 with a lot of designations, or if I decide to style a very inefficient fort).

Will update with results of my next fort. For now I guess, I'll just forge a pair of boots for every-dwarf to sidestep the potential infection problem.


*Update*

Doesn't seem to have made any significant FPS improvements. Much less clutter though. Will tinker more.

kwieland

2011-04-13 09:20

reporter   ~0017337

I have an established fort (15+ years) 60 dwarfs, half kids. 3x3 embark area. No original clothes left, they've all rotted away. However, all my dwarfs are equiped with cloth cloaks (6), trousers, and socks. These items routinely wear. My normal is 150 FPS. All my dwarfs have rooms with chests and cabinets. Do yours?

My FPS goes down to 40-50 after a siege when I've got a lot of crap to pick up from ground level. I think the FPS hit is due to lots of pathing calculations. Somewhere I read that the path calculation is redone everytime a dwarf runs into another dwarf. Also, during "normal" times, lots of the dwarfs are constructing things, which doesn't call pathing.

My take: my FPS hit is due to pathing not wear.

user6

2011-04-13 14:25

  ~0017347

Reminder sent to: Orkel2, Ter13

This report will be resolved as "unable to reproduce" in the near future unless saves are uploaded that demonstrate that rotting clothes have a significant impact on FPS, independent of other factors.

devek

2011-04-18 17:22

reporter   ~0017434

@Ter13

>The idea that item count affects FPS is indicative of overall design issues.

>I am very certain that there is a bit of code running unintentionally due to the
>owned and broken clothing. I believe the dwarf is constantly seeking the nearest
>replacement for that broken clothing, even though he has a second bit of
>clothing already. I believe that this is a leading cause to FPS hits, and should
>be pretty easy to profile and confirm.

How can it be profiled and confirmed if no one ever bothers to upload relevant saves? I think that is more of a design issue with the user base :P

You don't even need Toady to do it, I can use memory hacking to unown all items and then delete them manually to see... if I had a save to work with that is :)

ag

2011-04-19 04:09

reporter   ~0017441

I tried unowning, dumping into a smasher and pulling the lever on all worn clothes with my new cleanowned tool in DFHack, and it gave at most 10-15% improvement on my save (quite near to the statistical noise level).

Dumping all the cheap junk my stonecrafter created over the years onto a caravan seemed to have a similar effect. The latter was actually inspired by some specific results oprofile gave me, namely by relatively large time spent in binary search to find the container item of an object.

Dekon

2011-04-20 20:51

reporter   ~0017452

Maybe in a few weeks once classes are all done I'll try and do a fort and collect a crap ton of goblin junk and have it set up for a save, so at least leave this open for a bit longer.

If I've heard correctly (and I may not be correct about this), items have to constantly check themselves to determine 'Am I muddy? Am I on fire? Am I needing to be moved to a stockpile/dump? Am I...' etc etc. Because they're constantly being checked for this, it creates an intense drop in FPS when items start to build dramatically simply because so many things are undergoing this process of self check so frequently.

ag

2011-05-01 02:41

reporter   ~0017591

After some more profiling and staring at assembly, I have an easy optimization suggestion: reimplement flagarrayst::has_flag as a self-contained inline function without any external calls. It is used a lot for quering flags in raw structures, and currently every usage results in a cross-dll call without any opportunity for the compiler to use the fact that the flag ID is known at compile time. In the linux version get_slot_and_addbit_uchar accounts for 50% of all time spent in libgraphics.so.

gmafk

2011-09-15 23:25

reporter   ~0018727

I think I have proof that clothing is a source of a significant drop in FPS. My fort (31.25, linux) started at 100 FPS and every migrant wave reduced it until it was running at 30-40 FPS with 120 dwarves.

Save: http://dffd.wimbli.com/file.php?id=4943

All 120 dwarves are gathered near the garbage dump (F5) and are idle. All armor, legwear, headwear, handwear and footwear is designated for dumping (but dwarves own it so they won't dump it).

Step 1: Do nothing for 10-20s and let the FPS counter stabilize (39 FPS)
Step 2: Run 'cleanowned all' in DFHack
Step 3: Watch your FPS rise as dwarves discard their clothing into lava (54 FPS). This step can take around 2 minutes - you can use the stocks menu to check how many items have been destroyed so far).

Improvement from 39 FPS (25.64 ms) to 54 FPS (18.51 ms) is pretty significant: 7.12 ms spent per frame just to process clothes.

Xen0n

2011-09-19 14:55

reporter   ~0018742

Perhaps, add a control in case it was just getting rid of so many items that gave the improvement, not necessarily clothes. Take the same save and dump an equal number of crafts (or rocks or something) and compare the FPS improvement to when you dumped all the clothes.

gmafk

2011-09-20 18:45

reporter   ~0018745

I used the save from my previous post: http://dffd.wimbli.com/file.php?id=4943

Items other than clothes can be easily tested using 'autodump destroy'. Unfortunately, it doesn't seem to work on equipped items, so I just copied results from my previous test for comparison.

Step 1: Mark items for dumping.
Step 2: Run 'autodump destroy' in DFHack.

The game starts at 39 FPS (25.64 ms) before each test. Results:

Destroying 1,7 k clothes: 54 FPS (18.51 ms), -7.12 ms
Destroying 1,7 k stone (limestone + gabbro): 40 FPS (25.00 ms), -0.64 ms
Destroying 4,5 k stone (all stone on the map): 42 FPS (23.80 ms), -1.83 ms

It seems that processing clothes takes a lot more time than processing rocks.

kwieland

2011-09-20 20:07

reporter   ~0018746

Last edited: 2011-09-20 20:10

 gmafk, good work. I think the clothes are PART of the issue, but I think there are other things as well (like contaminants, pathing). I was going to suggest abandoning and reclaiming, but there are other bugs there, so it would be hard to isolate the real cause.

The original bug report states:"Dwarves are constantly checking to see if they need new clothes once they own a broken piece of clothing, and this is causing excess processing to be done" Do you think this is the case? Wouldn't that make your FPS drop after you "liberate" the clothes? Have you run the liberated game for a year to see if the FPS stays up?

thvaz

2012-03-15 12:13

reporter   ~0021488

Last edited: 2012-03-18 00:24

 This one shouldn't be fixed by today's devlog?

Kogut

2012-03-18 07:56

reporter   ~0021554

It is hidden (workaround is to simply produce clothes) but probably if "dwarves wear clothing (we turned it off and tried it), the FPS steadily decreases as more and more dwarves wear out their clothing".

Toady One

2012-03-19 04:01

administrator   ~0021588

As far as I can tell from having a few hundred dwarves around and rotting their clothing and placing clothing piles and so on, this overall situation is a lot better. That's not to say there aren't going to be problems, especially the ongoing general issues from having lots of objects around, but I'm going to mark this one off.

Issue History

Date Modified Username Field Change
2011-02-10 15:53 Ter13 New Issue
2011-02-11 08:38 user6 Note Added: 0015082
2011-02-11 08:38 user6 Tag Attached: AWAITING UPDATE
2011-02-11 09:46 Ter13 Note Added: 0015086
2011-02-11 10:05 Ter13 Note Edited: 0015086
2011-02-14 15:10 Kogut Note Added: 0015104
2011-02-14 15:40 IT 000 Note Added: 0015105
2011-02-15 02:34 Kogut Note Added: 0015107
2011-02-16 11:17 hyndis Note Added: 0015122
2011-03-16 12:06 user6 Relationship added related to 0004207
2011-03-16 18:19 user6 Note Added: 0016309
2011-03-17 01:32 AbuDhabi Note Added: 0016318
2011-03-17 13:36 AbuDhabi Note Edited: 0016318
2011-03-30 14:21 Orkel2 Note Added: 0016816
2011-03-30 14:21 Orkel2 Note Edited: 0016816
2011-03-30 14:39 user6 Note Added: 0016818
2011-03-31 00:16 profit Note Added: 0016827
2011-03-31 00:39 profit Note Added: 0016830
2011-03-31 00:49 InsanityPrelude Note Added: 0016831
2011-03-31 01:03 AbuDhabi Note Added: 0016832
2011-03-31 08:01 Kogut Note Added: 0016850
2011-03-31 09:21 InsanityPrelude Note Added: 0016858
2011-03-31 09:32 user6 Relationship added related to 0002481
2011-04-09 08:40 Dekon Note Added: 0017232
2011-04-09 17:39 alcohol_dependent Note Added: 0017259
2011-04-09 17:41 alcohol_dependent Note Edited: 0017259
2011-04-09 18:22 alcohol_dependent Note Edited: 0017259
2011-04-09 19:26 alcohol_dependent Note Edited: 0017259
2011-04-10 10:31 alcohol_dependent Note Edited: 0017259
2011-04-13 09:20 kwieland Note Added: 0017337
2011-04-13 14:25 user6 Note Added: 0017347
2011-04-18 17:22 devek Note Added: 0017434
2011-04-19 04:09 ag Note Added: 0017441
2011-04-20 20:51 Dekon Note Added: 0017452
2011-05-01 02:41 ag Note Added: 0017591
2011-09-15 23:25 gmafk Note Added: 0018727
2011-09-19 14:55 Xen0n Note Added: 0018742
2011-09-20 18:45 gmafk Note Added: 0018745
2011-09-20 20:07 kwieland Note Added: 0018746
2011-09-20 20:09 kwieland Note Edited: 0018746
2011-09-20 20:10 kwieland Note Edited: 0018746
2012-03-15 12:13 thvaz Note Added: 0021488
2012-03-18 00:24 thvaz Note Edited: 0021488
2012-03-18 07:23 user11 Tag Detached: AWAITING UPDATE
2012-03-18 07:56 Kogut Note Added: 0021554
2012-03-19 04:01 Toady One Note Added: 0021588
2012-03-19 04:01 Toady One Status new => resolved
2012-03-19 04:01 Toady One Fixed in Version => Next Version
2012-03-19 04:01 Toady One Resolution open => fixed
2012-03-19 04:01 Toady One Assigned To => Toady One