• Check out the results of the Techtree Contest #19!
  • Listen to a special audio message from Bill Roper to the Hive Workshop community (Bill is a former Vice President of Blizzard Entertainment, Producer, Designer, Musician, Voice Actor) 🔗Click here to hear his message!
  • Read Evilhog's interview with Gregory Alper, the original composer of the music for WarCraft: Orcs & Humans 🔗Click here to read the full interview.
  • Create a void inspired texture for Warcraft 3 and enter Hive's 34th Texturing Contest: Void! Click here to enter!
  • The Hive's 22nd Icon Contest: Creep Abilities is now concluded, time to vote for your favourite set of icons! Click here to vote!

Stable Lua FileIO

This is an update to the FileIO library, originally created by @Trokkin. It allows you to write strings into text files in the CustomMapData subfolder in the Warcraft 3 directory, then read from those text files in a later game. For documentation, see the original thread.

While working with FileIO, I experienced crashes from loading certain files, so I set out to investigate and fix them.

The Lua FileIO works by enclosing every string segment, which has a maximum length of 255, in long string delimiters [[ and ]]. This can get messed up in several ways:
  • If ]] appears inside the string, it prematurely closes the string and the game crashes upon loading the file. Therefore, each ] is replaced by !].
  • Several characters are escaped with a backslash when written to the text file. This will increase the size of the string by 1 for each of those characters, which, in turn, can make it exceed the string size limit of 255, cutting off the closing delimiter, and therefore crashing the game when trying to load the file. Therefore, this version counts the number of characters that need escaping and reduces the string size accordingly.
  • Some issues regarding \x25 and \\\\ that I don't fully understand, which I fixed through experimentation.
In addition, I removed the OnInit wrapper because it is possible to use the Save function in the Lua root without problems. Bear in mind, though, that using FileIO.Load from within the Lua root will cause the game to crash.

Lua:
if Debug then Debug.beginFile "FileIO" end
--[[
    FileIO v2

    Author: Trokkin
    Updated by Antares

    Provides functionality to read and write files, optimized with lua functionality in mind.

    API:

        FileIO.Save(filename, data)
            - Write string data to a file

        FileIO.Load(filename) -> string?
            - Read string data from a file. Returns nil if file doesn't exist.

        FileIO.SaveAsserted(filename, data, onFail?) -> bool
            - Saves the file and checks that it was saved successfully.
              If it fails, passes (filename, data, loadResult) to onFail.

    Optional requirements:
        DebugUtils by Eikonium                          @ https://www.hiveworkshop.com/threads/330758/

    Inspired by:
        - TriggerHappy's Codeless Save and Load         @ https://www.hiveworkshop.com/threads/278664/
        - ScrewTheTrees's Codeless Save/Sync concept    @ https://www.hiveworkshop.com/threads/325749/
        - Luashine's LUA variant of TH's FileIO         @ https://www.hiveworkshop.com/threads/307568/post-3519040
        - HerlySQR's LUA variant of TH's Save/Load      @ https://www.hiveworkshop.com/threads/331536/post-3565884

    Updated: 15 June 2025
--]]
do
    local RAW_PREFIX = ']]i([['
    local RAW_SUFFIX = ']])--[['
    local RAW_SIZE = 255 - #RAW_PREFIX - #RAW_SUFFIX
    local LOAD_ABILITY = FourCC('ANdc')
    local LOAD_EMPTY_KEY = '!@#$, empty data'

    local name

    local function open(filename)
        name = filename
        PreloadGenClear()
        Preload('")\nendfunction\n//!beginusercode\nlocal p={} local i=function(s) table.insert(p,s) end--[[')
    end

    local function write(s)
        --Avoid ]] in string.
        s = s:gsub("\x25]", "!]")
        --Repeated backslash literals are not processed correctly by the Preload natives.
        s = s:gsub("\\", "!\\")
        local i = 1
        local size = #s
        local pos, __, num, str
        repeat
            pos = i + RAW_SIZE - 1
            str = s:sub(i, pos)
            --Backslashes increase preload string size in file, causing the suffix to get swallowed, which in turn causes a crash.
            __, num = str:gsub("\\", "")
            pos = pos - num
            --A ] at the end of a preload string will crash the game. A newline character at the beginning will be swallowed.
            while s:sub(pos, pos) == "]" or s:sub(pos + 1, pos + 1) == "\n" do
                pos = pos - 1
            end
            Preload(RAW_PREFIX .. s:sub(i, pos) .. RAW_SUFFIX)
            i = pos + 1
        until i > size
    end

    local function close()
        Preload(']]BlzSetAbilityTooltip(' ..
            LOAD_ABILITY .. ', table.concat(p), 0)\n//!endusercode\nfunction a takes nothing returns nothing\n//')
        PreloadGenEnd(name)
        name = nil
    end

    ---@param filename string
    ---@param data string
    local function savefile(filename, data)
        open(filename)
        write(data)
        close()
    end

    ---@param filename string
    ---@return string?
    local function loadfile(filename)
        local s = BlzGetAbilityTooltip(LOAD_ABILITY, 0)
        BlzSetAbilityTooltip(LOAD_ABILITY, LOAD_EMPTY_KEY, 0)
        Preloader(filename)
        local loaded = BlzGetAbilityTooltip(LOAD_ABILITY, 0)
        BlzSetAbilityTooltip(LOAD_ABILITY, s, 0)
        if loaded == LOAD_EMPTY_KEY then
            return nil
        end
        Preloader("")
        loaded = loaded:gsub("!\x25]", "]"):gsub("!\\\\", "\\")
        return loaded
    end

    ---@param filename string
    ---@param data string
    ---@param onFail function?
    ---@return boolean
    local function saveAsserted(filename, data, onFail)
        savefile(filename, data)
        local res = loadfile(filename)
        if res == data then
            return true
        end
        if onFail then
            onFail(filename, data, res)
        end
        return false
    end

    FileIO = {
        Save = savefile,
        Load = loadfile,
        SaveAsserted = saveAsserted
    }
end
Contents

Stable Lua FileIO (Binary)

Reviews
Wrda
Simple, clear and efficient library. Hopefully there's no more edge cases :) Approved
Oh! Good one.
I worked on this issue some time ago and developed a different approach to search for [[ and increase the output to [=[ and further until it doesn't find the part in the chunk that's to be written. I think it's a more elegant solution than to shield brackets in-data back and forth. I haven't got to test it thoroughly so I didn't update my post, but I'll send it to you if you're interested to maintain it.
 
Oh! Good one.
I worked on this issue some time ago and developed a different approach to search for [[ and increase the output to [=[ and further until it doesn't find the part in the chunk that's to be written. I think it's a more elegant solution than to shield brackets in-data back and forth. I haven't got to test it thoroughly so I didn't update my post, but I'll send it to you if you're interested to maintain it.
That's also a valid option, but I don't think there's anything wrong with just masking the ]] temporarily. On average, there'll be fewer additional characters you need to insert, the logic becomes simpler, and it's faster as you can do everything in one gsub call.

If you have any more fixes or improvements that we should add, of course I'd be interested in them.
 
I had to add another replacement. \n literals seemed to get confused with newline characters, so I masked all newline characters and restored them later on. The most confusing thing is that this wasn't deterministic. Sometimes the \n literals were converted into linebreaks, sometimes they weren't. :peasant-thinking:
 
After some more testing, I found that repeated backslash literals are the only offending character sequence and the other replacements are not necessary (except for the square brackets). For whatever reason, both \\ and \\\\ result in the same output, so we mask the backslashes with a ! to split them apart.
 
In addition, I removed the OnInit wrapper because it is possible to use the Save function in the Lua root without problems. Bear in mind, though, that using FileIO.Load from within the Lua root will cause the game to crash.
OnInit.root exists, and also you can swap load function past some Init phase like so:
Lua:
OnInit.root("FileIO", function(require)
    ...

    ---@param filename string
    ---@return string?
    local function denyloadfile(filename)
        print("|cFF000000Attempt at loading the file [" .. filename .. "] too soon, make sure you're attempting to load files at the soonest at Trigger initialization phase")
        return ""
    end

    FileIO = {
        Save = savefile,
        Load = denyloadfile,
        SaveAsserted = saveAsserted
    }
   
    OnInit.final(function()
        FileIO.Load = loadfile
    end)
end)

PS: I just randomly decided to put Trigger Initialization phase and OnInit.final, I don't actually know at what phase it's safe to load.
 
I had some need of very large files that can be saved and sometimes loaded and was afraid of performance reduction, so solved it another way - I replaced the unsupported characters with unprintable characters, and only escaped the unprintables (which are rarely used). Also added an option to save the file as non-loadable, where only null terminator needs to be escaped, and where you can remove the usercode stuff.
 
Back
Top