[vJASS] FileDump - dump data to disk

[Aniki]

FileDump - dump data to disk

Using the Preload* natives it is possible to generate "preload" scripts (.pld) but we can also use those natives to dump some data to disk.

// if we run the following code
//
call PreloadGenClear()
call PreloadGenStart()
call Preload("Hello jassy-world")
call PreloadGenEnd("save\\my-file-name.pld")

// and we open <path-to-war3>\save\my-file-name.pld we should see:
function PreloadFiles takes nothing returns nothing

    call Preload( "Hello jassy-world" )
    call PreloadEnd( 0.0 )

endfunction

So it turns out the call to the Preload(...) native becomes exactly that... a call to
the call Preload( "Hello jassy-world" ) preload native. But that's kind of inconvenient.
We probably want only Hello jassy-world, the text that we wrote to appear, and not have the rest.

That's why FileDump comes with a simple Perl script (feel free to rewrite it in your favoirite language, except Python )
which searches for the preload calls, extracts only the text we wrote, and saves it into another file.

FileDump.j

//! novjass

FileDump has a very simple API:

path is the file path to the file we want to write to
we can have nested directories, eg:
    "save\\my-dumps\\dump-1.jass-dump"
and the Preload natives will create them if they do not exist

static method open takes string path returns Fd


calls open with the default path: "save" + PS + "output.jass-dump"
where PS is the PATH_SEPARATOR, which might be different for Mac OS users??

static method open_default takes nothing returns Fd


warns if the line is longer than MY_OS_MAX_PATH_LENGTH in DEBUG_MODE

method writeln takes string line returns nothing


creates/overwrites the file with the lines we passed to writeln
NOTE: if this method is not called nothing will be written to disk

method close takes nothing returns nothing

//! endnovjass


library FileDump

globals
    private constant string MY_OS_PATH_SEPARATOR = "\\"

    // warn if we try to write more than that many characters per line
    private constant integer MY_OS_MAX_PATH_LENGTH = 260
endglobals

struct Fd
    static string PS = MY_OS_PATH_SEPARATOR

    // we use the save directory by default because it exists (and we don't want to pollute the
    // war3 directory with our own directory)
    // and it should be less cluttered compared to the other directories, because
    // it should only contain other directories and no files, which we might overwrite...
    private static string DEFAULT_FILE = "save" + PS + "output.jass-dump"

    private static hashtable file_lines = InitHashtable()
    private static thistype ef_fd

    private integer lines_count = 0
    private integer line_index = 0
    readonly string path = DEFAULT_FILE

    static method open takes string path returns thistype
        local thistype this = thistype.allocate()
        set this.path = path
        return this
    endmethod

    static method open_default takes nothing returns thistype
        return thistype.allocate()
    endmethod

    method writeln takes string str returns nothing
        local integer str_len = StringLength(str)

static if DEBUG_MODE then
        if str_len > MY_OS_MAX_PATH_LENGTH then
            call BJDebugMsg("|cffFF0000warning: could not write all characters of string \"" + SubString(str, 0, 64) + "\"... with length " + I2S(str_len) + " to file " + this.path + "|r")
        endif
endif

        call SaveStr(file_lines, this, lines_count, str)
        set lines_count = lines_count + 1
    endmethod

    private static method dump_exec takes nothing returns nothing
        local integer MAX_LINES_PER_EXEC = 5000
        local thistype fd = ef_fd
        local integer i
        local string str
        local integer str_len

        set i = 1
        loop
            exitwhen i > MAX_LINES_PER_EXEC
            set str = LoadStr(file_lines, fd, fd.line_index)

            call Preload(str)

            set fd.line_index = fd.line_index + 1
            if fd.line_index == fd.lines_count then
                return
            endif

            set i = i + 1
        endloop

        call ExecuteFunc(dump_exec.name)
    endmethod

    private method dump takes nothing returns nothing
        set ef_fd = this
        call ExecuteFunc(dump_exec.name)
    endmethod

    method close takes nothing returns nothing
        call PreloadGenClear()
        call PreloadGenStart()
            call dump()
        call PreloadGenEnd(path)

        call this.destroy()
    endmethod
endstruct

endlibrary

jass-fd.pl:

use strict;
use warnings;

my $args_count = @ARGV;
my @files;
my $delete_source_files = '';

# parse args
{
    my $abort = '';

    for my $arg (@ARGV) {
        if (substr($arg, 0, 2) eq '--') {
            my $option_name = substr($arg, 2);
            if ($option_name eq 'delete-source-files') {
                $delete_source_files = 1;
            }
            else {
                $abort = 1;
                print "error: unknown option: '--$option_name'\n";
            }
        }
        elsif (!-e $arg) {
            $abort = 1;
            print "error: file does not exist: '$arg'\n";
        }
        else {
            push(@files, $arg);
        }
    }

    if ($abort) {
        exit(1);
    }
}

# extract the lines from the supplied files
{
    my $PATTERN_START = 'call Preload( "';
    my $start_offset = length($PATTERN_START);
    my $PATTERN_END = '" )';

    for my $in_file_name (@files) {
        open(my $in_file, '<', $in_file_name) || die($!);

        my $out_file_name = substr($in_file_name, 0, rindex($in_file_name, '.'));
        $out_file_name .= ".txt";
        open(my $out_file, '>', $out_file_name) || die($!);

        for (;;) {
            my $line = readline($in_file);
            last if !defined($line);

            # we want to find the lines matching:
            # '  call Preload( "<data>" )'
            # and extract the data

            my $i = index($line, $PATTERN_START);
            my $j = rindex($line, $PATTERN_END);
            if ($i != -1 && $j != -1) {
                my $start_index = $i + $start_offset;
                my $chars_count = $j - $start_index;
                my $data = substr($line, $start_index, $chars_count);

                print {$out_file} $data, "\n";
            }
        }
    }
}

if ($delete_source_files) {
    for my $file (@files) {
        unlink($file) || die($!);
    }
}

demo

library demo initializer init requires FileDump

globals
    integer array heroes
    constant integer MAX_HEROES = 24
    integer max_heroes
endglobals

function heroes_init takes nothing returns nothing
    set heroes[1] = 'Hamg'
    set heroes[2] = 'Hmkg'
    set heroes[3] = 'Hpal'
    set heroes[4] = 'Hblm'

    set heroes[5] = 'Ofar'
    set heroes[6] = 'Otch'
    set heroes[7] = 'Obla'
    set heroes[8] = 'Oshd'

    set heroes[9] = 'Udea'
    set heroes[10] = 'Ulic'
    set heroes[11] = 'Udre'
    set heroes[12] = 'Ucrl'

    set heroes[13] = 'Edem'
    set heroes[14] = 'Ekee'
    set heroes[15] = 'Emoo'
    set heroes[16] = 'Ewar'

    set heroes[17] = 'Nalc'
    set heroes[18] = 'Nbst'
    set heroes[19] = 'Nbrn'
    set heroes[20] = 'Nfir'
    set heroes[21] = 'Nngs'
    set heroes[22] = 'Npbm'
    set heroes[23] = 'Nplh'
    set heroes[24] = 'Ntin'

    set max_heroes = MAX_HEROES
endfunction

struct MaxHeroStats
    integer str = 0
    integer str_hero_id

    integer agi = 0
    integer agi_hero_id

    integer int = 0
    integer int_hero_id

    integer all_stats = 0
    integer all_stats_hero_id
endstruct
globals
    MaxHeroStats array max_stats[10]
endglobals

function get_hero_name takes integer hero_id returns string
    local unit u = CreateUnit(Player(0), hero_id, 0.0, 0.0, 0.0)
    local string result = GetUnitName(u)
    call RemoveUnit(u)
    set u = null
    return result
endfunction

function get_heroes_stats takes nothing returns nothing
    local Fd fd
    local unit u
    local MaxHeroStats ms
    local integer hero_index
    local integer hero_id
    local integer level
    local integer str
    local integer agi
    local integer int
    local integer stats_sum
    local string line
    local integer MAX_LEVEL = 10

    set level = 1
    loop
        exitwhen level > MAX_LEVEL
        set ms = MaxHeroStats.create()
        set max_stats[level] = ms

        set hero_index = 1
        loop
            exitwhen hero_index > max_heroes
            set hero_id = heroes[hero_index]
            set u = CreateUnit(Player(0), hero_id, 0.0, 0.0, 0.0)
            call SetHeroLevelBJ(u, level, false)

            set str = GetHeroStr(u, /*include-bonuses*/ false)
            set agi = GetHeroAgi(u, /*include-bonuses*/ false)
            set int = GetHeroInt(u, /*include-bonuses*/ false)
            set stats_sum = str + agi + int
            call RemoveUnit(u)

            if ms.str < str then
                set ms.str = str
                set ms.str_hero_id = hero_id
            endif

            if ms.agi < agi then
                set ms.agi = agi
                set ms.agi_hero_id = hero_id
            endif

            if ms.int < int then
                set ms.int = int
                set ms.int_hero_id = hero_id
            endif

            if ms.all_stats < stats_sum then
                set ms.all_stats = stats_sum
                set ms.all_stats_hero_id = hero_id
            endif

            set hero_index = hero_index + 1
        endloop

        set level = level + 1
    endloop

    // Save the stats
    set fd = Fd.open("save" + Fd.PS + "hero-stats.jass-dump")

    set level = 1
    loop
        exitwhen level > MAX_LEVEL
        set ms = max_stats[level]

        set line = "[Lvl " + I2S(level) + "]: "

        set line = line + "max-str: " + get_hero_name(ms.str_hero_id) + "(" + I2S(ms.str) + "); "
        set line = line + "max-agi: " + get_hero_name(ms.agi_hero_id) + "(" + I2S(ms.agi) + "); "
        set line = line + "max-int: " + get_hero_name(ms.int_hero_id) + "(" + I2S(ms.int) + "); "
        set line = line + "max-stats: " + get_hero_name(ms.all_stats_hero_id) + "(" + I2S(ms.all_stats) + "); "
        call fd.writeln(line)

        set level = level + 1
    endloop

    // We must call close, otherwise nothing will be written to disk
    call fd.close()

    set u = null
endfunction

function init takes nothing returns nothing
    call ExecuteFunc(heroes_init.name)
    call ExecuteFunc(get_heroes_stats.name)
endfunction

endlibrary

If we run the demo in WE we should have a the file <path-to-war3>\save\hero-stats.jass-dump
with the content:

function PreloadFiles takes nothing returns nothing

    call Preload( "[Lvl 1]: max-str: Crypt Lord(26); max-agi: Blademaster(23); max-int: Naga Sea Witch(22); max-stats: Blademaster(57); " )
    call Preload( "[Lvl 2]: max-str: Crypt Lord(29); max-agi: Blademaster(24); max-int: Naga Sea Witch(25); max-stats: Blademaster(62); " )
    call Preload( "[Lvl 3]: max-str: Crypt Lord(32); max-agi: Blademaster(26); max-int: Naga Sea Witch(28); max-stats: Blademaster(68); " )
    call Preload( "[Lvl 4]: max-str: Crypt Lord(35); max-agi: Blademaster(28); max-int: Naga Sea Witch(31); max-stats: Blademaster(74); " )
    call Preload( "[Lvl 5]: max-str: Crypt Lord(38); max-agi: Blademaster(30); max-int: Naga Sea Witch(34); max-stats: Blademaster(81); " )
    call Preload( "[Lvl 6]: max-str: Crypt Lord(42); max-agi: Blademaster(31); max-int: Lich(37); max-stats: Blademaster(86); " )
    call Preload( "[Lvl 7]: max-str: Crypt Lord(45); max-agi: Blademaster(33); max-int: Lich(40); max-stats: Blademaster(92); " )
    call Preload( "[Lvl 8]: max-str: Crypt Lord(48); max-agi: Blademaster(35); max-int: Lich(43); max-stats: Blademaster(98); " )
    call Preload( "[Lvl 9]: max-str: Crypt Lord(51); max-agi: Blademaster(37); max-int: Lich(47); max-stats: Blademaster(105); " )
    call Preload( "[Lvl 10]: max-str: Crypt Lord(54); max-agi: Blademaster(38); max-int: Lich(50); max-stats: Blademaster(110); " )
    call PreloadEnd( 0.0 )

endfunction

And if we run the above perl script with:
perl jass-fd.pl hero-stats.jass-dump
we should get another file called: hero-stats.txt with the content:

[Lvl 1]: max-str: Crypt Lord(26); max-agi: Blademaster(23); max-int: Naga Sea Witch(22); max-stats: Blademaster(57);
[Lvl 2]: max-str: Crypt Lord(29); max-agi: Blademaster(24); max-int: Naga Sea Witch(25); max-stats: Blademaster(62);
[Lvl 3]: max-str: Crypt Lord(32); max-agi: Blademaster(26); max-int: Naga Sea Witch(28); max-stats: Blademaster(68);
[Lvl 4]: max-str: Crypt Lord(35); max-agi: Blademaster(28); max-int: Naga Sea Witch(31); max-stats: Blademaster(74);
[Lvl 5]: max-str: Crypt Lord(38); max-agi: Blademaster(30); max-int: Naga Sea Witch(34); max-stats: Blademaster(81);
[Lvl 6]: max-str: Crypt Lord(42); max-agi: Blademaster(31); max-int: Lich(37); max-stats: Blademaster(86);
[Lvl 7]: max-str: Crypt Lord(45); max-agi: Blademaster(33); max-int: Lich(40); max-stats: Blademaster(92);
[Lvl 8]: max-str: Crypt Lord(48); max-agi: Blademaster(35); max-int: Lich(43); max-stats: Blademaster(98);
[Lvl 9]: max-str: Crypt Lord(51); max-agi: Blademaster(37); max-int: Lich(47); max-stats: Blademaster(105);
[Lvl 10]: max-str: Crypt Lord(54); max-agi: Blademaster(38); max-int: Lich(50); max-stats: Blademaster(110);

I.e only what we wrote with the .writeln method calls.

PS: Blademaster IMBA!

 

[Almia]

http://www.hiveworkshop.com/forums/...1-a-234246/?prev=u=Almia&r=20&status=a&page=2

)
Tho my resource doesn't support multiple instances and only works on Windows machine

 

[Aniki]

lol, didn't know the .bat parser/interpreter was this forgiving =)...

only works on Windows machine...

It's funny because I think perl comes pre-installed with Mac OS X and Windows does not, but FileDump works on windows but it might not work on OSX (unless the Preload natives convert Windows\\style\\paths to OSX/style/paths automatically, or the user properly sets the MY_OS_PATH_SEPARATOR and uses it when calling open set fd = Fd.open("save" + Fd.PS + "my-dumps" + Fd.PS + "out.jass-dumo") =)

Also it seemed that for big log files Debug I/O's generated .bat files take a bit of time to run, because the .bat interpreter has to report the incorrect syntax, I think you can use the REM (line comment thingy) to reduce those by half?

 

[IcemanBo]

There exists JASS/File IO.w3x at master · nestharus/JASS · GitHub.

 

[Flux]

There exists JASS/File IO.w3x at master · nestharus/JASS · GitHub.

Technically it is not uploaded to this site but Nestharus' File IO was working fine last time I used it so I don't see a reason to create this new one.