View Issue Details

Jump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
0010459Dwarf FortressTechnical -- Generalpublic2017-12-17 13:392018-01-05 04:21
ReporterSuspriscus Assigned ToToady One  
PrioritynormalSeveritymajorReproducibilityrandom
Status resolvedResolutionfixed 
PlatformWindowsOSWindows 10 
Product Version0.44.02 
Fixed in Version0.44.04 
Summary0010459: Game stuttering/freezing every couple of steps
DescriptionI have a few saves where every couple of steps, the game completely freezes for about half a second. This occurs on saves with otherwise good FPS, and occurs seemingly at random. I have only been able to "fix" this by returning to previous saves where no stuttering was present and continuing playing as I had done before.
Steps To ReproduceNo idea how to reliably reproduce.

The following contains two saves on the same world (the same as save no. 2, additional info) made roughly 8 minutes apart, to try and replicate the stuttering issue.

http://dffd.bay12games.com/file.php?id=13317

The first, earlier save has no problem with stuttering, whereas the second, later save does. Another attempt was made to recreate the stuttering problem by going back to the first of these test saves and simply letting it run as before, but this was unsuccessful.
Additional InformationCopies of the saves that have the stuttering problem:
http://dffd.bay12games.com/file.php?id=13314 - Save no. 1
http://dffd.bay12games.com/file.php?id=13315 - Save no. 2
http://dffd.bay12games.com/file.php?id=13316 - Save no. 3

TagsSave Included

Relationships

related to 0009109 resolvedToady One Massive stutter every few seconds 
related to 0010444 resolvedToady One Extreme Lag and Stuttering after 7 Year Old Fort on World with Default Settings 
related to 0009554 resolvedlethosor Game freezes for about 10 seconds then runs for 10 seconds repeatedly 
related to 0008076 acknowledgedlethosor Game starts to stutter really badly 

Activities

lethosor

2017-12-17 15:14

manager   ~0037346

See also https://www.reddit.com/r/dwarffortress/comments/7kal8b/biweekly_df_questions_thread/drd3orx/ - doesn't seem utility-dependent.

Sarmatian123

2017-12-18 14:34

reporter   ~0037350

Last edited: 2017-12-18 14:39

 Reproducible : Not always random.

It seems I always construct fortress in 5-10 years with 20k+ blocks, so my fps drops even with 100 Dwarves to 15(15) fps. So what my staggering low fps have to do with finding some game mechanics causing such staggering freezes? Well, I had entire games turning into staggering frames, as well as staggering, which refuse to let be for entire season, or short periods of such stagger. They were out of blue sky to me. However at few occasions I noticed some particular events taking place for entire duration of staggering.

This event is combat. Maybe my game did not progress much. Lagged. Staggered. Froze even. However one thing grew very fast and with lots of pages. This was combat report. There were like 20+ pages over time long like few seconds for many characters involved.

I can not be 100% sure on the cause of all fps stagger. For sure combat at low fps, can sometimes cause it. Also thing to keep in mind, even if we don't dig into caverns/lava/hell, there are still animals spawning there and pathing could cause low fps too.

sionlife

2017-12-19 06:21

reporter   ~0037354

Last edited: 2017-12-19 06:40

 looks like duplicate of

http://www.bay12games.com/dwarves/mantisbt/view.php?id=8076
http://www.bay12games.com/dwarves/mantisbt/view.php?id=9109

As a note, I have also experienced this stutter phenomenon on forts in 44.02 that have reached 200 populations. Not as bad as the OP but I get a stutter of about 0.5-1s every 5s or so. It is very noticeable. I don't remember having quite so many problems on 43.05.

sionlife

2017-12-20 08:59

reporter   ~0037363

I have found that if you retire the fortress and then reclaim it back, it cures the stuttering problem. You will lose all your drinks though.

lethosor

2017-12-21 19:24

manager   ~0037370

sionlife: it's hard to tell for sure if those are duplicates, since their causes are unclear. I've marked this as related to 0009109. The save in 0008076 has disappeared, so I closed it for now.

lethosor

2017-12-21 19:25

manager   ~0037371

Reminder sent to: Sarmatian123, Suspriscus

Can you check on the size of your errorlog.txt and gamelog.txt files? Are they significantly increasing when stuttering happens?

Sarmatian123

2017-12-22 04:09

reporter   ~0037373

Last edited: 2017-12-22 04:34

 I never check those files, unless something crashes. DF is very stable. It crashed for me only recently on 0.43.05 on 2 occasions. One was directly after starting new embark. I just continued from save and it was magically fine then. Second was after I was messing with different fertilization settings (seasonal vs fertilization) for growing crops on a field. This is not on the topic though... Sorry!

I do check however the "a" and "r" menus. This is how I discovered during couple of those staggering periods some noticeable increases in combat reports. So no, size of those files are increasing only with how long I played given DF version with one exception.

0.42.04 errorlog.txt is significantly larger (around 300kB), then entire 0.43.05 (around 200kb), which I played for significantly longer time with more embarks, each lasting longer as well. It is also the version I was experiencing the significant 10 sec freeze, 10 sec stagger slo-mo play without quantum stockpiles. There are lots of messages about failed pathing, mostly of my Dwarven Military. Also about double tagging items in food (barrel) stockpiles. I used to have horror of 10 sec freeze and 10 sec slow staggering on 42.04, but converting at least stone and log stockpiles into quantum stockpiles solved it for me. It really did help to lessen the pain of staggering. Nothing else worked, but quantum stockpiles. At least this is what I've thought about it so far. Could this be an issue with military Dwarves dragging their stone haul with them into combat/exercise? Nothing else coming to my mind in this combination of quantum stockpiles, stone+logs, military Dwarves and pathing. These days I use also multiple feeder-stockpiles with 3 wheelbarrows each to avoid military Dwarves slo-mo dragging stones when going on duty to station or to train in barracks. Though dumping stones from inside of wheelbarrows is an irritating activity. Also it helps to evacuate quickly all Dwarves from surface, when siege starts. To stop "a" menu spam, I just need to forbid the wheelbarrows, that were left outside. These days I run all my stockpiles as quantum with exception of food ones.

Also I had this 10 sec freeze and 10 sec stagger slo-mo play on a pocket world, but with all other medium settings. Currently I run only all-low settings with my embarks. Maybe this is also a difference? Nothing else comes really to my mind from my experiences playing vanilla DF for last 2 years. Yes, vanilla client. 10-15(10-15) fps here after putting 20k+ blocks in, like a clockwork. Old computer here and can't really afford upgrading it either...

Should I upload the errorlog.txt for 0.42.04? There are only 2 types of failure messages there really. Examples here bellow:
(1)
path fail: dwarf,54,90,169 -> 71,70,11: Id 0000083:Path Goal Seek Station:Station Squad Move at 72,73,11
path fail: dwarf,Construct Building,59,71,171 -> 60,73,171: Id 0000196:Path Goal Main Job Building:Station Mill Building at 58,78,170
path fail: elk bird,67,136,115 -> 68,135,114: Id 0001290:Path Goal Seek Station Flood:Station Owner at 68,135,114
loop path fail: <same square> dwarf,Dump Item,63,59,169 -> 64,60,169: Id 0000677:Path Goal Seek Item for Storage:Station Mill Building at 37,88,171
path fail: troglodyte,66,103,115 -> 56,102,114: Id 0001688:Path Goal Marauder Mill:Station Marauder Mill at 56,102,114
(2)
DOUBLE TAGGED ITEM: Store Item in Stockpile, quarry bush leaves [2]
DOUBLE TAGGED ITEM: Store Item in Stockpile, *pecan wood barrel <0000005>*
DOUBLE TAGGED ITEM: Store Item in Stockpile, (steel anvil)

EDIT: This 0.42.04 errorlog.txt is significantly larger then overage. I have however even earlier version 0.40.24. I didn't use there quantum stockpiles either and I think the significant stagger there was too. All errors in 0.40.24 error.log.txt are double tagged items though or solemnly loop path fails. Mostly double tagged items though. Also I have only 1 save there, the last save of my first ever embark. Later on, I was saving region directory more frequently... for reasons... :)

sionlife

2017-12-22 08:30

reporter   ~0037374

reply to lethosor latest comment:

I cleared both my errorlog.txt and gamelog.txt files and ran the stuttering for a few minutes. I rechecked both files and the errorlog.txt was still empty and the gamelog.txt had just 5 lines. So I don't think the problem is being reported in these files.

Sarmatian123

2017-12-22 10:30

reporter   ~0037376

I will agree with sionlife on this one. Gamelog.txt is not reporting on unseen combat from not discovered yet caverns and I've seen combat "r"-menu growing during stuttering moments only from combat I could see with my own eyes.

The big stutter of 10 sec screen freeze + 10 sec of stutter afterward is lessened, if not solved, with quantum stockpiles. I have no clue why. It just works like that in DF.

The stone hauling seems to be just an inconvenience. I fail to see how it can cause stutter. Though error flood in errorlog.txt seems somehow slightly correlated with this inconvenience. For some reason.

Suspriscus

2017-12-22 11:42

reporter   ~0037377

Cleared errorlog.txt and gamelog.txt, then ran each save for a few minutes.

Save 1:

errorlog, 1KB: Two lines about double tagged items, and one about a path fail.

gamelog, 16KB: Combat between a forgotten beast and another creature in a cavern. A lot of dwarf vomiting. The combat stopped, stuttering continued.

Save 2:

errorlog, 0KB: Nothing

gamelog, 75KB: Nothing for a few minutes, then a lot of lines of "combat" from dwarfs training. Again, stuttering didn't seem correlated to this.

Save 3:

errorlog, 0KB: Nothing

gamelog, 7KB: Nothing for the first few minutes, then a dwarf starts hunting a wild boar. Again, doesn't seem correlated to the stuttering.

Sarmatian123

2017-12-22 16:18

reporter   ~0037379

@Suspriscus,
Did you discover all 3 caverns? They tend to be at lv80, lv40 and lv30.
Did you uncover vision in entirety of all those 3 caverns?

On my embarks reports about forgottens' arrival I got only, when they entered tile I had vision on. I never seen any predators in discovered lava sea and hell does not fight hell. Also...

Did hell get road to surface/cavern/lava in some other embark in the same world? I never let those roaches out. Inside there is nothing worth digging for anyhow.

lethosor

2017-12-22 16:27

manager   ~0037380

I asked because I saw someone on Reddit report a log that was around a gigabyte in size, and was having similar issues to this. Thanks for checking on that - this issue, at least, doesn't appear to be due to that, then.

Suspriscus

2017-12-22 18:04

reporter   ~0037381

@Sarmatian123

I uncovered all three caverns in my first and second saves, not all revealed. In the third I didn't discover the caverns at all, though.

GraphicH

2017-12-23 12:42

reporter   ~0037388

Another save showing this issue:
http://dffd.bay12games.com/file.php?id=13339

I have season backups, the issue began about year 503 and got progressively worse until finally getting into an unplayable state in 504.

thvaz

2017-12-26 20:30

reporter   ~0037406

Another save, for 0.44.03 http://dffd.bay12games.com/file.php?id=13349

My computer should handle this fortress without any problem, but the game is stuttering heavily and sometimes it stops responding (but it recovers after a minute or two)

Sarmatian123

2017-12-31 00:01

reporter   ~0037448

Last edited: 2017-12-31 00:44

 For me severe stuttering is solved by using quantum stockpiles. Dunno why or how it works, but it works.

I still get freezes of screen during combat.

However my fps got halved and almost unnoticeable light stutter appeared, when I allowed 10 hunters hunt freely and to have in my 110 Dwarves fortress 20-30 military Dwarves training constantly (8 squads, training schedule alike "10-2-2-2-2-2-10-2-2-2-2-2").

Now, that I was pointed at those message files in DF directory, I took notice about something strange. Appearance of light stutter is combined with strange limitation of announcements on screen. Announcements from being few pages long got short to barely 6-8 lines of latest messages. All caverns are still hidden on my embark and I discovered only small part of lava sea.


There is a bug from 2010. A minor issue.
http://www.bay12games.com/dwarves/mantisbt/view.php?id=1430
It seems announcements and reports share common stack of 3000 messages.

However when I looked in reports and they almost all were empty too. Like announcements. Where messages of announcements and reports go?

Those txt files on harddrive are not growing exponentially either. Why there is light stutter? I understand combat/sparing/hunting could be causing this light stutter, but why then the quantum stockpiles are magically solving the most heavy and severe case of it? How quantum stockpiles are related to sparing&hunting?

EDIT:
MY BAD! GAMELOG.TXT exploded! It grew suddenly to 22.5mb. HUGE! Like 80% of all of its content now are the sparing messages. It loads into my mousepad text editor like for entire 5 minutes.

Sarmatian123

2017-12-31 00:40

reporter   ~0037449

Last edited: 2017-12-31 01:11

 WOW!

I renamed gamelog.txt. Created new 0 size gamelog.txt. Started DF anew. My fps magically went up from 7(7)fps to 13(13)fps. just like that. 14(14)-17(17) is what I had in fps when I finished construction of my 20k+ blocks fortress.

I checked on announcements... and there were those few last announcements still on.
I checked on reports... and there were those few reports still on.

However I looked every single of those 4 sparing entries and it seems each was 8-9 pages long and one 18 pages long.

The light stutter still on. Dwarves still sparring ;)

EDIT:
GAMELOG.TXT is being written with speed of 1-3kb/1sec.
10 military Dwarves activity: spar.

Also my DF speed went down again to around 8-13 fps. It varies. It is higher then before with flat 7(7) fps. Light stutter as well as sparing still continues. :) 15(15)-16(16) fps! Removing old 22.5mb gamelog.txt was huge to boost fps in my DF.

PS.

How quantum stockpiles are related to sparing? Is it errorlog.txt thingy? So without quantum stockpiles my errorlog.txt would be few gb huge by now?

Orkel

2018-01-01 16:16

reporter   ~0037461

Here's my save: http://dffd.bay12games.com/file.php?id=13370
At only 73 dwarves, my FPS suddenly dropped to 50 from 140, and there is heavy stutter. Like, it is not smooth at all, the movement of dwarves is jagging heavily.
I have done everything I can think of to get it back up again, like killing every single animal and pet on the map (including the caverns using dfhack), making sure no doors are locked, removing all spatters on the map, killing wounded dwarves in their beds, blocking the flow of the brook and underground water using obsidian so they don't flow out of the map, deleting the gamelog.txt, atomsmashing corpses, killing all visitors to the fortress, etc.

Sarmatian123

2018-01-01 17:34

reporter   ~0037463

Orkel: Did you turn all, except food stuff, stockpiles into quantum stockpiles and then delete errorlog.txt too?

Orkel

2018-01-02 13:55

reporter   ~0037475

Last edited: 2018-01-02 13:55

 Not quantum stockpiles, but I deconstructed all workshops and stockpiles as a whole (didn't fix it).

Even if I made quantum stockpiles it is unlikely to be the cause, since I've run bigger forts with more FPS and much, much bigger stockpiles before this fort.

Sarmatian123

2018-01-02 16:03

reporter   ~0037477

http://dwarffortresswiki.org/index.php/DF2014:Maximizing_framerate#Fortress_Design

"The quantity of items in any particular stack doesn't affect framerate so much as the number of stacks in general, due to the resultant impact on hauling, stockpiles, pathfinding and other CPU-intensive tasks."

As you can read above from Wiki's quote, deconstructing stockpiles and workshops are not the same as using quantum stockpiles.

I would suggest, you use "d-b-d" to mass mark items for dump with "i"-menu designated 1x1 dump area. Simple even for a beginner to quantum stockpiling. You will need later to allow those items too.

Then read on Wiki about quantum stockpiles and build some wooden minecarts and using one or two 11x11 feeder stockpiles create "b-c-S" track stops for automated minecart auto-dump quantum stockpiling.

Quantum Stockpiling, generally speeds your frame rate, even if you still experience stutter.

I wish there was some designation to mark entrance(s) in a burrow for the pathfinding algorithm. It would save on some cpu. I doubt pathfinding algorithm even uses burrow designations currently. I wish we could assign cats to burrows, like we can with Dwarves. I do not think killing pest outsidefortress makes sense. Pest auto-regenerates. I hope there is some coded limit to the pest levels.

sionlife

2018-01-02 19:11

reporter   ~0037479

Last edited: 2018-01-02 19:14

 I would rather the cause of the stuttering be found and fixed rather than for everybody to start using QSP and brush the problem under the carpet.

In any case, I have seen an example of low fps happen right after the embark in the first 10 seconds. There are no stockpiles and just the items on the wagon. QSP is not the answer we are looking for here.

Sarmatian123

2018-01-02 21:14

reporter   ~0037480

Once (40.24 or 42.04 it was I think) I have seen my fps stop almost dead after breaching a cave... The stutter I had was nightmarish. I was under impression my computer is crashing. Then miraculously fps came back to life after I atom smashed under a bridge 3 times contents of time after time completely filled 11x11 refuse. Then I had repeat of the same after breaching 2nd cave. My cats were very busy then. I even suggested that maybe vermin has something to do with this stutter/low fps. Now, I do not even breach caves any more and am happy if I can play at 10+ fps.

There are plenty of issues causing low fps. You can read on Wiki how to maximize the frame rate. Stutter you always will have with combat, hunting and sparing anyhow. So better to have light stutter, then unplayable stutter. Unplayable stutter is caused by far more issues then just those mentioned by me.

It seems DF gameplay begs for optimization and smoothing rough corners on some of those glitches. This is why this bug/issue won't be fixed any time soon. Surely not in this year. However now folks should know to delete those log.txt files in df directory to get at least some fps back. Besides QSP of course. :)

RudeGuyGames

2018-01-04 16:38

reporter   ~0037508

Here's a save with a new embark that has "inherited" the stuttering. I have tried creating a new world, and starting a new embark there got rid of the stuttering for the time being, so it seems to be a world-specific issue.

http://dffd.bay12games.com/file.php?id=13387

Toady One

2018-01-05 04:21

administrator   ~0037510

Found two new problems that fixed the really bad stuttering on the saves I downloaded here; from e.g. five second pauses to smooth play. There are still some minor older issues (for instance, hf births on the day can load files and cause a little tick, and it's not straightforward to change that, but that was in 0.43 and earlier versions).

Issue History

Date Modified Username Field Change
2017-12-17 13:39 Suspriscus New Issue
2017-12-17 15:13 lethosor Tag Attached: Save Included
2017-12-17 15:14 lethosor Note Added: 0037346
2017-12-17 15:14 lethosor Assigned To => lethosor
2017-12-17 15:14 lethosor Status new => acknowledged
2017-12-18 14:34 Sarmatian123 Note Added: 0037350
2017-12-18 14:39 Sarmatian123 Note Edited: 0037350
2017-12-19 06:21 sionlife Note Added: 0037354
2017-12-19 06:40 sionlife Note Edited: 0037354
2017-12-20 08:59 sionlife Note Added: 0037363
2017-12-21 19:21 lethosor Relationship added related to 0009109
2017-12-21 19:24 lethosor Note Added: 0037370
2017-12-21 19:25 lethosor Note Added: 0037371
2017-12-21 19:25 lethosor Status acknowledged => feedback
2017-12-21 19:27 lethosor Relationship added related to 0010444
2017-12-21 19:31 lethosor Relationship added related to 0009554
2017-12-22 04:09 Sarmatian123 Note Added: 0037373
2017-12-22 04:34 Sarmatian123 Note Edited: 0037373
2017-12-22 08:30 sionlife Note Added: 0037374
2017-12-22 10:30 Sarmatian123 Note Added: 0037376
2017-12-22 11:42 Suspriscus Note Added: 0037377
2017-12-22 11:42 Suspriscus Status feedback => assigned
2017-12-22 16:18 Sarmatian123 Note Added: 0037379
2017-12-22 16:27 lethosor Note Added: 0037380
2017-12-22 16:27 lethosor Status assigned => acknowledged
2017-12-22 18:04 Suspriscus Note Added: 0037381
2017-12-23 12:42 GraphicH Note Added: 0037388
2017-12-25 15:03 lethosor Relationship added related to 0008076
2017-12-26 20:30 thvaz Note Added: 0037406
2017-12-31 00:01 Sarmatian123 Note Added: 0037448
2017-12-31 00:04 Sarmatian123 Note Edited: 0037448
2017-12-31 00:04 Sarmatian123 Note Edited: 0037448
2017-12-31 00:14 Sarmatian123 Note Edited: 0037448
2017-12-31 00:25 Sarmatian123 Note Edited: 0037448
2017-12-31 00:26 Sarmatian123 Note Edited: 0037448
2017-12-31 00:40 Sarmatian123 Note Added: 0037449
2017-12-31 00:41 Sarmatian123 Note Edited: 0037449
2017-12-31 00:42 Sarmatian123 Note Edited: 0037449
2017-12-31 00:44 Sarmatian123 Note Edited: 0037448
2017-12-31 00:52 Sarmatian123 Note Edited: 0037449
2017-12-31 00:53 Sarmatian123 Note Edited: 0037449
2017-12-31 00:54 Sarmatian123 Note Edited: 0037449
2017-12-31 00:55 Sarmatian123 Note Edited: 0037449
2017-12-31 00:56 Sarmatian123 Note Edited: 0037449
2017-12-31 00:59 Sarmatian123 Note Edited: 0037449
2017-12-31 01:11 Sarmatian123 Note Edited: 0037449
2018-01-01 16:16 Orkel Note Added: 0037461
2018-01-01 17:34 Sarmatian123 Note Added: 0037463
2018-01-02 13:55 Orkel Note Added: 0037475
2018-01-02 13:55 Orkel Note Edited: 0037475
2018-01-02 16:03 Sarmatian123 Note Added: 0037477
2018-01-02 19:11 sionlife Note Added: 0037479
2018-01-02 19:14 sionlife Note Edited: 0037479
2018-01-02 21:14 Sarmatian123 Note Added: 0037480
2018-01-04 16:38 RudeGuyGames Note Added: 0037508
2018-01-05 04:21 Toady One Note Added: 0037510
2018-01-05 04:21 Toady One Status acknowledged => resolved
2018-01-05 04:21 Toady One Fixed in Version => Next Version
2018-01-05 04:21 Toady One Resolution open => fixed
2018-01-05 04:21 Toady One Assigned To lethosor => Toady One