Inform7 slow compiling Windows 10 (windows defender problem)

Authoring Inform 7
21

Just a hunch but is part of the problem that so many of the files created have “suspicious” file extensions (something the AV software thinks might be a vector for malware)? Could “auto.inf” be created with a different extension and perhaps could the index pages be created as xml (albeit with changed content) or with some non-standard extension (could be the same content, but IDEs might need to be updated to treat the files as before)?

EDIT: I tried to hack on ni.exe with a hexeditor to experiment with the above to some extent but it did not not seem to like the changes

22

I’ve already tried this: it doesn’t help. The problem is the generation of .html files for the index etc. However, regardless of what they’re called, Windows Defender still notices that they’re HTML and very slowly checks them. So far there seems to be no way round this other than to manually configure Windows Defender to ignore the executables that generate the HTML.

23

Not knowing how Window Defender works, I’m wondering if the index files were produced more smartly, would WD check them faster? I’ve already raised this at inform7.uservoice.com/forums/57 … -very-slow

Having thousands of inline scripts and inline css makes the files very large and slow to render, but those thousands of scripts may also make WD process them slowly too. If the JS code was collected into one code block (and rewritten so that it doesn’t need to specifically invoke each individual rule or object etc) then it might make a big difference.

This would be a change for Graham of course.

1 Like
24

I don’t really know how Windows software works, so this might be an unhelpful suggestion, but would it be possible to defer the index file generation or do it in another thread so that it wouldn’t block the compilation? If the files were created in the background the author wouldn’t have to wait for Windows Defender to check them.

25

It is possible, though I can’t imagine there’s much enthusiasm to rewrite the HTML generation on the off-chance that Windows Defender behaves better as a result. It would be a good thing to do from the point of view of generating better HTML, and if that helped the Defender problems that would be a bonus, but I don’t expect this to be a high priority.

A thread per-se isn’t the answer, as Inform 7 works like many compiler front-ends: there is a command line process (“ni.exe”) that does all the work, including generating the index: it is this process that Windows Defender is slowing down, since the front-end waits for it to complete. It might, in principle, be feasible to have ni.exe write the Inform 6 generated code first, then signal to the front-end that the next step could start while it generated the index. However, that requires considerable thought, and imposes an on-going cost in terms of ni.exe having to honour its new contract to always generate the Inform 6 code first. I am considering it, but don’t expect anything any time soon.

26

I’m resurrecting this thread with my recent successful experience (February 6th 2021) because this is still very much an issue; the best way to deal with it seems to have changed; and it’s very simple but not clearly indicated in this thread although it was one of the first two threads I found when searching on this issue. Using Windows 10, my compiles were very slow, and I could tell it was Windows Defender because Task Manager revealed that “Antimalware Service Executable” (MsMpEng.exe) was very triggered by compiles.

Most of the information you find when googling how to disable Windows Defender did not work for me, either because it’s out of date or specific to some systems. The one piece of advice that worked for me was installing a different piece of antivirus software. I installed the free Avira (obviously others may work) and instantly (without even a reboot) my compile time was down by probably a whole order of magnitude, and certainly from “I definitely going to need to alt-tab to something else” to “I’m definitely not going to need to”.

1 Like
27

Here’s a current Windows Security (what I think is the new version of Defender) solution for those who’d rather not replace their entire antivirus over this:

  • Open Windows Security
  • Select Virus and Threat Protection
  • Click “Manage Settings” under Virus & Threat Protection Settings
  • Scroll down and click “Add or remove exclusions”

You’ll be at an exclusion manager now. Add the Inform7 installation folder, the Inform folder in your Documents, and your project folders to the exclusions. Personally, I saved all my Inform projects into one folder and just added that folder, rather than each project folder individually. This solved the problem for me.

1 Like
28

Thanks a lot! I have had this problem for a while and never figured out how to actually add the exclusions (I was confused because Windows Defender still exists, but it seems to just handle firewall stuff now).

29

I haven’t used Inform7 much at all, but I have tried it and noticed the surprisingly long build times. I’m a software engineer who’s seen these kinds of bad interactions between compilers and anti-malware software a lot. I know that sometimes a relatively simple change in how a program produces files can eliminate the bottleneck. I’d love to help everyone facing these slowdowns.

I downloaded Inform7 to my Windows 10 machine, and confirmed that, with a trivial story, the compile time is less than a second but about 10 seconds according to the clock on the wall. I also confirmed that turning off the “real time” Windows antimalware software completely eliminated the delay.

Next I used Sysinternals Process Monitor to watch what ni.exe (the Inform compiler) and MsMpEng.exe (the MS malware scanner) are up to. It’s strictly a file I/O bottleneck: neither application is hitting the network nor is either spending excessive time on registry lookups.

Other than ni.exe loading up a disconcertingly antique version of the CRT, it seemed well behaved. But even in ProcMon, it was easy to see that the output of each HTML index file was momentarily blocked as MsMpEng scanned the previously emitted one. But it wasn’t quite clear why it was blocked.

So I took an event trace (using Bruce Dawson’s UI4ETW) and tried to learn more in Windows Performance Analyzer. (I needed more practice with WPA anyway.)

My first observation is that the Windows search indexer is doing far more scanning of the files ni.exe is creating than MsMpEng, and yet it doesn’t seem to have any direct impact on the speed of ni.exe (assuming you’ve got a spare core for it to run on).

Eventually I found that, somehow, ni.exe gets blocked trying to create each file while MsMpEng scans the previous one. None of the delays are particularly large, but there are just over 100 of them, and they add up to about 9 seconds, which is consistent with the observations.

But it didn’t make sense to me how one process scanning the previously created file could block creation of the next file. It seems like those should be able to run in parallel.

So I loaded up the symbol files (PDBs) and tried to look at the stacks. Fortunately, Inform7 provides the PDBs. Unfortunately, they are in a very very old format that WPA doesn’t support, so ni.exe stacks are completely opaque, which prevented me from making much more progress. (The old PDB format is consistent with the very old version of the C run-time library.)

I found a 2019 article that said Inform was going to be open sourced, so I hoped to see if I could recompile ni.exe to get a modern symbol file. Alas, the open sourcing of Inform isn’t yet complete.

My best guess is that the very old CRT library may be blocking file i/o on some global(ish) lock because the library wasn’t very thread savvy back then. I know other security software installs filesystem drivers that can slow down file i/o for all processes, but MsMpEng seems to be a plain old user-mode scanner that uses standard i/o calls, so I’m not sure why it would have this kind of impact when the search indexer doesn’t.

Once Inform7’s repo is made public, I think we’ll be able to figure out the root cause. Hopefully, we can update how the Windows version is built so that it’ll be more amenable to analysis tools.

In the mean time, it should be enough to grant some exclusions in the Virus & Thread Protection Settings.