• 🏆 Texturing Contest #33 is OPEN! Contestants must re-texture a SD unit model found in-game (Warcraft 3 Classic), recreating the unit into a peaceful NPC version. 🔗Click here to enter!

[vJASS] base64

Level 13
Joined
Nov 7, 2014
Messages
571
base64 - encode bytes to base 64 and then decode strings and integers

General information about Base64.

base64 requires either Binary or Bitwise libraries.
base64 also requires either chrord or Ascii.

API

JASS:
// Must be called before writing new bytes
base64.rewind takes nothing returns nothing

// There's a limit on how much bytes can be written
// if this limit is exceeded returns false (failure), otherwise if all
// of str's bytes were written returns true (success)
base64.write_str takes string str returns boolean

// writes the 4 bytes of the integer
// if successful returns true
// else returns false
base64.write_int takes integer i returns boolean

// returns the bytes written encoded as base64
base64.encode takes nothing returns string

// decodes the bytes in the encoded string
base64.decode takes string base64_encoded_string returns nothing

// reads bytes_count number of bytes and returns a string
base64.read_str takes integer bytes_count returns string

// reads 4 bytes and returns an integer
base64.read_int takes nothing returns integer

// reads 1 bytes and returns an integer
base64.read_byte takes nothing returns integer

Example usage:

JASS:
        call base64.rewind()
        call base64.write_str("Foo Bar")
        call base64.write_int(0x85ABCDEF)
        set base64_encoded_str = base64.encode()

        call BJDebugMsg(base64_encoded_str) // Rm9vIEJhcoWrze8=

        call base64.decode(base64_encoded_str)
        set my_str = base64.read_str(7) // length: 7
        set my_int = base64.read_int()

        call BJDebugMsg(my_str) // Foo Bar
        call BJDebugMsg(I2S(my_int)) // -2052338193


        call base64.rewind()
        call base64.write_int('Hpal')

        call base64.decode(base64.encode())
        call BJDebugMsg(base64.read_str(4)) // Hpal


        call base64.rewind()
        call base64.write_int('hfoo')

        call base64.decode(base64.encode())
        call BJDebugMsg(I2S(base64.read_byte())) // 104

JASS:
library base64 uses /*
 */ optional Binary /* [url]http://www.wc3c.net/showthread.php?t=101855[/url] by d07.RiV
 */ optional Bitwise, /* [url]http://www.hiveworkshop.com/forums/jass-resources-412/snippet-bitwise-249223/[/url] by Nestharus, Magtheridon96 & Bannar
 */ optional chrord, /* [url]http://www.hiveworkshop.com/forums/submissions-414/chrord-274579/[/url]
 */ optional Ascii /* [url]http://www.hiveworkshop.com/forums/jass-resources-412/snippet-ascii-190746/[/url] */

struct base64 extends array

    // JASS_MAX_STRING_LENGTH = 4100
    // [url]http://www.hiveworkshop.com/forums/lab-715/documentation-string-type-240473/#post2441361[/url]
    //
    // base64-digits-count = 4 * ceil(byte-count  / 3)
    // 4100 = 4 * ceil(3073 / 3)
    //
    static constant integer MAX_WRITE_BYTE_COUNT = 3073


    // utils
    //
    static method floor_pos takes real real_pos returns integer
        return R2I(real_pos)
    endmethod

    static method ceil_pos takes real real_pos returns integer
        local integer int_pos = R2I(real_pos)

        if real_pos == int_pos then
            return int_pos
        endif

        return int_pos + 1
    endmethod

    static method round_up_to_nearest_multiple_of takes real value, integer n returns integer
        return ceil_pos(value / n) * n
    endmethod

    static method round_down_to_nearest_multiple_of takes real value, integer n returns integer
        return floor_pos(value / n) * n
    endmethod

    static method get_padding takes integer bytes_count returns integer
        return round_up_to_nearest_multiple_of(bytes_count, 3) - bytes_count
    endmethod

    static method b64_ord takes string s returns integer
static if LIBRARY_chrord then
        return ord(s)
elseif LIBRARY_Ascii then
        return Char2Ascii(s)
else
        MissingRequiredLibrary
endif
    endmethod

    static method b64_chr takes integer i returns string
static if LIBRARY_chrord then
        return chr(i)
elseif LIBRARY_Ascii then
        return Ascii2Char(i)
else
        MissingRequiredLibrary
endif
    endmethod

    static method b64_SHL takes integer i, integer shift returns integer
static if LIBRARY_Binary then
        return SHL(i, shift)
elseif LIBRARY_Bitwise then
        return Bitwise.shiftl(i, shift)
else
        MissingRequiredLibrary
endif
    endmethod

    static method b64_SHR takes integer i, integer shift returns integer
static if LIBRARY_Binary then
        return SHR(i, shift)
elseif LIBRARY_Bitwise then
        return Bitwise.shiftr(i, shift)
else
        MissingRequiredLibrary
endif
    endmethod

    static method b64_AND takes integer a, integer b returns integer
static if LIBRARY_Binary then
        return AND(a, b)
elseif LIBRARY_Bitwise then
        return Bitwise.AND32(a, b)
else
        MissingRequiredLibrary
endif
    endmethod

    static string array digits

    static method onInit takes nothing returns nothing
        set digits[0] = "A"
        set digits[1] = "B"
        set digits[2] = "C"
        set digits[3] = "D"
        set digits[4] = "E"
        set digits[5] = "F"
        set digits[6] = "G"
        set digits[7] = "H"
        set digits[8] = "I"
        set digits[9] = "J"
        set digits[10] = "K"
        set digits[11] = "L"
        set digits[12] = "M"
        set digits[13] = "N"
        set digits[14] = "O"
        set digits[15] = "P"
        set digits[16] = "Q"
        set digits[17] = "R"
        set digits[18] = "S"
        set digits[19] = "T"
        set digits[20] = "U"
        set digits[21] = "V"
        set digits[22] = "W"
        set digits[23] = "X"
        set digits[24] = "Y"
        set digits[25] = "Z"
        set digits[26] = "a"
        set digits[27] = "b"
        set digits[28] = "c"
        set digits[29] = "d"
        set digits[30] = "e"
        set digits[31] = "f"
        set digits[32] = "g"
        set digits[33] = "h"
        set digits[34] = "i"
        set digits[35] = "j"
        set digits[36] = "k"
        set digits[37] = "l"
        set digits[38] = "m"
        set digits[39] = "n"
        set digits[40] = "o"
        set digits[41] = "p"
        set digits[42] = "q"
        set digits[43] = "r"
        set digits[44] = "s"
        set digits[45] = "t"
        set digits[46] = "u"
        set digits[47] = "v"
        set digits[48] = "w"
        set digits[49] = "x"
        set digits[50] = "y"
        set digits[51] = "z"
        set digits[52] = "0"
        set digits[53] = "1"
        set digits[54] = "2"
        set digits[55] = "3"
        set digits[56] = "4"
        set digits[57] = "5"
        set digits[58] = "6"
        set digits[59] = "7"
        set digits[60] = "8"
        set digits[61] = "9"
        set digits[62] = "+"
        set digits[63] = "/"
    endmethod

    static method digit_ordinal_to_6_bit_value takes integer o returns integer
        if 0x41 <= o then
            // 'A' .. 'Z'
            if o <= 0x5A then
                set o = o - 0x41
                return o

            // 'a' .. 'z'
            else // elseif o <= 0x7A then
                set o = o - 0x47
                return o
            endif

        elseif 0x30 <= o then
            // '='
            if o == 0x3D then
                return 0

            // '0' .. '9'
            else // elseif o <= 0x39 then
                set o = o + 4
                return o
            endif

        // '+'
        elseif o == 0x2B then
            return 62

        // '/'
        else // elseif o == 0x2F then
            return 63
        endif
    endmethod


    static integer array enc_buf[thistype.MAX_WRITE_BYTE_COUNT]
    // points to the position of the next byte to write to
    static integer write_ptr = 0

    static integer array dec_buf
    // points to the position of the next byte to read from
    static integer read_ptr = 0

    static method rewind takes nothing returns nothing
        local integer i

        set i = 0
        loop
            exitwhen i >= write_ptr
            set enc_buf[i] = 0
            set i = i + 1
        endloop

        set write_ptr = 0
    endmethod

    static method write_str takes string str returns boolean
        local integer str_length = StringLength(str)
        local integer i

        if write_ptr + str_length > MAX_WRITE_BYTE_COUNT then
            // could not write all of str's bytes
            return false
        endif

        set i = 0
        loop
            exitwhen i >= str_length
            set enc_buf[write_ptr + i] = b64_ord(SubString(str, i, i + 1))
            set i = i + 1
        endloop

        set write_ptr = write_ptr + str_length

        // all of str's bytes were written
        return true
    endmethod

    static method write_int takes integer i returns boolean
        local integer b3
        local integer b2
        local integer b1
        local integer b0
        local integer carry

        if write_ptr + 4 > MAX_WRITE_BYTE_COUNT then
            return false
        endif

static if LIBRARY_Binary then
       if i > 0 then
            set b3 = SHR(AND(i, 0x7F000000), 24)
            set b2 = SHR(AND(i, 0x00FF0000), 16)
            set b1 = SHR(AND(i, 0x0000FF00), 8)
            set b0 = AND(i, 0x000000FF)

        elseif i < 0 then
            if i != 0x80000000 then
                set i = -i

                set b0 = AND(i, 0x000000FF)
                if b0 == 0 then
                    set carry = 1
                else
                    set b0 = 0xFF - b0 + 1
                    set carry = 0
                endif

                set b1 = SHR(AND(i, 0x0000FF00), 8)
                if b1 != 0 or carry == 0 then
                    set b1 = 0xFF - b1 + carry
                    set carry = 0
                endif

                set b2 = SHR(AND(i, 0x00FF0000), 16)
                if b2 != 0 or carry == 0 then
                    set b2 = 0xFF - b2 + carry
                    set carry = 0
                endif

                set b3 = SHR(AND(i, 0x7F000000), 24)
                set b3 = 0xFF - b3 + carry

            else
                set b3 = 0x80
                set b2 = 0x00
                set b1 = 0x00
                set b0 = 0x00
            endif

        else // if i == 0
            set b3 = 0x00
            set b2 = 0x00
            set b1 = 0x00
            set b0 = 0x00
        endif

elseif LIBRARY_Bitwise then
        set b3 = b64_SHR(b64_AND(i, 0xFF000000), 24)
        if b3 < 0 then
            set b3 = b3 + 0x100
        endif
        set b2 = b64_SHR(b64_AND(i, 0x00FF0000), 16)
        set b1 = b64_SHR(b64_AND(i, 0x0000FF00), 8)
        set b0 = b64_AND(i, 0x000000FF)

else
        MissingRequiredLibrary
endif

        set enc_buf[write_ptr] = b3
        set enc_buf[write_ptr + 1] = b2
        set enc_buf[write_ptr + 2] = b1
        set enc_buf[write_ptr + 3] = b0

        set write_ptr = write_ptr + 4

        return true
    endmethod

    static method encode takes nothing returns string
        local string result = ""
        local integer bytes_count = write_ptr
        local integer iterations_count
        local integer i
        local integer byte_offset
        local integer a_24_bits
        local integer digit_index
        local integer padding

        // each iteration we convert 3 bytes to 4 base 64 digits
        set iterations_count = round_down_to_nearest_multiple_of(bytes_count, 3) / 3

        set i = 0
        set byte_offset = 0
        loop
            exitwhen i >= iterations_count

            set a_24_bits = 0
            set a_24_bits = a_24_bits + b64_SHL(enc_buf[byte_offset], 16)
            set a_24_bits = a_24_bits + b64_SHL(enc_buf[byte_offset + 1], 8)
            set a_24_bits = a_24_bits + enc_buf[byte_offset + 2]

            // 0xFC0000 = 0x3F << 18
            // 0x3F000 = 0x3F << 12
            // 0xFC0 = 0x3F << 6
            // 0x3F = 0x3F << 0
            // 0x3F = 0b111_111

            set digit_index = b64_SHR(b64_AND(a_24_bits, 0xFC0000), 18)
            set result = result + digits[digit_index]

            set digit_index = b64_SHR(b64_AND(a_24_bits, 0x3F000), 12)
            set result = result + digits[digit_index]

            set digit_index = b64_SHR(b64_AND(a_24_bits, 0xFC0), 6)
            set result = result + digits[digit_index]

            set digit_index = b64_AND(a_24_bits, 0x3F)
            set result = result + digits[digit_index]

            set i = i + 1
            set byte_offset = byte_offset + 3
        endloop

        set padding = get_padding(bytes_count)

        if padding == 1 then
            set a_24_bits = 0
            set a_24_bits = a_24_bits + b64_SHL(enc_buf[byte_offset], 16)
            set a_24_bits = a_24_bits + b64_SHL(enc_buf[byte_offset + 1], 8)
            // set a_24_bits = a_24_bits + 0 // 0 pad

            set digit_index = b64_SHR(b64_AND(a_24_bits, 0xFC0000), 18)
            set result = result + digits[digit_index]

            set digit_index = b64_SHR(b64_AND(a_24_bits, 0x3F000), 12)
            set result = result + digits[digit_index]

            set digit_index = b64_SHR(b64_AND(a_24_bits, 0xFC0), 6)
            set result = result + digits[digit_index]

            // set digit_index = b64_AND(a_24_bits, 0x3F)
            // set result = result + digits[digit_index]
            set result = result + "="

        elseif padding == 2 then
            set a_24_bits = 0
            set a_24_bits = a_24_bits + b64_SHL(enc_buf[byte_offset], 16)
            // set a_24_bits = a_24_bits + 0 // 0 pad
            // set a_24_bits = a_24_bits + 0 // 0 pad

            set digit_index = b64_SHR(b64_AND(a_24_bits, 0xFC0000), 18)
            set result = result + digits[digit_index]

            set digit_index = b64_SHR(b64_AND(a_24_bits, 0x3F000), 12)
            set result = result + digits[digit_index]

            // set digit_index = b64_SHR(b64_AND(a_24_bits, 0xFC0), 6)
            // set result = result + digits[digit_index]
            set result = result + "="

            // set digit_index = b64_AND(a_24_bits, 0x3F)
            // set result = result + digits[digit_index]
            set result = result + "="
        endif

        return result
    endmethod

    static method decode takes string str returns boolean
        local integer digits_count = StringLength(str)
        local integer iterations_count
        local integer i
        local integer digit_offset
        local integer a_24_bits
        local integer byte_ord
        local integer byte_offset
        local integer padding

        if digits_count == 0 then
            return false
        endif

        set read_ptr = 0

        // each iteration we convert 4 base 64 digits to 3 bytes
        // set iterations_count = round_down_to_nearest_multiple_of(digits_count, 4) / 4
        // the encoder always produces a str with a length multiple of 4
        // (padded with 1 ('=') or 2 ('==') equal signs.
        set iterations_count = digits_count / 4

        set padding = 0
        if SubString(str, digits_count - 2, digits_count - 1) == "=" then
            set padding = padding + 1
        endif
        if SubString(str, digits_count - 1, digits_count) == "=" then
            set padding = padding + 1
        endif

        if padding != 0 then
            set iterations_count = iterations_count - 1
        endif

        set i = 0
        set digit_offset = 0
        set byte_offset = 0
        loop
            exitwhen i >= iterations_count

            set a_24_bits = 0
            set a_24_bits = a_24_bits + b64_SHL(digit_ordinal_to_6_bit_value(b64_ord(SubString(str, digit_offset, digit_offset + 1))), 18)
            set a_24_bits = a_24_bits + b64_SHL(digit_ordinal_to_6_bit_value(b64_ord(SubString(str, digit_offset + 1, digit_offset + 2))), 12)
            set a_24_bits = a_24_bits + b64_SHL(digit_ordinal_to_6_bit_value(b64_ord(SubString(str, digit_offset + 2, digit_offset + 3))), 6)
            set a_24_bits = a_24_bits + digit_ordinal_to_6_bit_value(b64_ord(SubString(str, digit_offset + 3, digit_offset + 4)))

            set byte_ord = b64_SHR(b64_AND(a_24_bits, 0xFF0000), 16)
            set dec_buf[byte_offset] = byte_ord

            set byte_ord = b64_SHR(b64_AND(a_24_bits, 0x00FF00), 8)
            set dec_buf[byte_offset + 1] = byte_ord

            set byte_ord = b64_AND(a_24_bits, 0x0000FF)
            set dec_buf[byte_offset + 2] = byte_ord

            set i = i +1
            set digit_offset = digit_offset + 4
            set byte_offset =  byte_offset + 3
        endloop

        if padding == 1 then
            set a_24_bits = 0
            set a_24_bits = a_24_bits + b64_SHL(digit_ordinal_to_6_bit_value(b64_ord(SubString(str, digit_offset, digit_offset + 1))), 18)
            set a_24_bits = a_24_bits + b64_SHL(digit_ordinal_to_6_bit_value(b64_ord(SubString(str, digit_offset + 1, digit_offset + 2))), 12)
            set a_24_bits = a_24_bits + b64_SHL(digit_ordinal_to_6_bit_value(b64_ord(SubString(str, digit_offset + 2, digit_offset + 3))), 6)
            // set a_24_bits = a_24_bits + 0 // 0 pad

            set byte_ord = b64_SHR(b64_AND(a_24_bits, 0xFF0000), 16)
            set dec_buf[byte_offset] = byte_ord

            set byte_ord = b64_SHR(b64_AND(a_24_bits, 0x00FF00), 8)
            set dec_buf[byte_offset + 1] = byte_ord

            // set byte_ord = b64_AND(a_24_bits, 0x0000FF)
            set dec_buf[byte_offset + 2] = 0

        elseif padding == 2 then
            set a_24_bits = 0
            set a_24_bits = a_24_bits + b64_SHL(digit_ordinal_to_6_bit_value(b64_ord(SubString(str, digit_offset, digit_offset + 1))), 18)
            set a_24_bits = a_24_bits + b64_SHL(digit_ordinal_to_6_bit_value(b64_ord(SubString(str, digit_offset + 1, digit_offset + 2))), 12)
            // set a_24_bits = a_24_bits + 0 // 0 pad
            // set a_24_bits = a_24_bits + 0 // 0 pad

            set byte_ord = b64_SHR(b64_AND(a_24_bits, 0xFF0000), 16)
            set dec_buf[byte_offset] = byte_ord

            // set byte_ord = b64_SHR(b64_AND(a_24_bits, 0x00FF00), 8)
            set dec_buf[byte_offset + 1] = 0

            // set byte_ord = b64_AND(a_24_bits, 0x0000FF)
            set dec_buf[byte_offset + 2] = 0
        endif

        return true
    endmethod

    static method read_str takes integer bytes_count returns string
        local string result = ""
        local integer i

        set i = 0
        loop
            exitwhen i >= bytes_count

            set result = result + b64_chr(dec_buf[read_ptr])
            set read_ptr = read_ptr + 1

            set i = i + 1
        endloop

        return result
    endmethod

    static method read_int takes nothing returns integer
        local integer result = 0

        set result = result + b64_SHL(dec_buf[read_ptr], 24)
        set result = result + b64_SHL(dec_buf[read_ptr + 1], 16)
        set result = result + b64_SHL(dec_buf[read_ptr + 2], 8)
        set result = result + dec_buf[read_ptr + 3]

        set read_ptr = read_ptr + 4

        return result
    endmethod

    static method read_byte takes nothing returns integer
        local integer result = dec_buf[read_ptr]
        set read_ptr = read_ptr + 1
        return result
    endmethod

endstruct

endlibrary

Edit: switched from Binary to Bitwise
Edit2: restored support for Binary =)
 

Attachments

  • base64-demo.w3x
    40.7 KB · Views: 191
Last edited:
Level 31
Joined
Jul 10, 2007
Messages
6,306
Hm.....

Why this over a more general library? What's important about using radix 64?


It seems like you're trying to do some sort of save/load thing? : o


Also, the Binary lib you are using can't handle 32 bits. It can only handle 31. Just making sure you know that. We do have some binary libs on the hive that can handle 32 bits : ).

You might have read this post and seen that Binary from wc3c is faster for 32 bits.

http://www.hiveworkshop.com/forums/jass-resources-412/snippet-bitwise-249223/

I say that this is only the case because Binary form wc3c doesn't actually handle 32 bits ; P.
 
Last edited:
Level 13
Joined
Nov 7, 2014
Messages
571
There was indeed a bug in base64.write_int() for negative numbers.

I agree that Bitwise seems to be more correct but I did manage to teach Binary to handle negative numbers (in the case that I need it to):

JASS:
        // set number = 0x81000000
        // set number = 0x810000EF
        // set number = 0x8100CD00
        // set number = 0x8100CDEF
        // set number = 0x81AB0000
        // set number = 0x81AB00EF
        // set number = 0x81ABCD00
        set number = 0x81ABCDEF

if false then
        call BJDebugMsg("Bitwise")

        set b3 = Bitwise.shiftr(Bitwise.AND32(number, 0xFF000000), 24)
        if b3 < 0 then
            set b3 = b3 + 0x100
        endif
        set b2 = Bitwise.shiftr(Bitwise.AND32(number, 0x00FF0000), 16)
        set b1 = Bitwise.shiftr(Bitwise.AND32(number, 0x0000FF00), 8)
        set b0 = Bitwise.AND32(number, 0x000000FF)

else
        call BJDebugMsg("Binary")

       if number > 0 then
            set b3 = SHR(AND(number, 0x7F000000), 24)
            set b2 = SHR(AND(number, 0x00FF0000), 16)
            set b1 = SHR(AND(number, 0x0000FF00), 8)
            set b0 = AND(number, 0x000000FF)

        elseif number < 0 then
            if number != 0x80000000 then

                set b0 = -number
                set b0 = AND(b0, 0x000000FF)
                if b0 != 0 then
                    set b0 = 0x100 - b0
                endif

                set b1 = -number
                set b1 = AND(b1, 0x0000FF00)
                set b1 = SHR(b1, 8)
                if b1 != 0 then
                    if b0 != 0 then
                        set b1 = 0xFF - b1
                    else
                        set b1 = 0x100 - b1
                    endif
                endif

                set b2 = -number
                set b2 = AND(b2, 0x00FF0000)
                set b2 = SHR(b2, 16)
                if b2 != 0 then
                    if b1 != 0 or b0 != 0 then
                        set b2 = 0xFF - b2
                    else
                        set b2 = 0x100 - b2
                    endif
                endif

                set b3 = -number
                set b3 = SHR(b3, 24)

                if b2 != 0 or b1 != 0 or b0 != 0 then
                    set b3 = 0xFF - b3
                else
                    set b3 = 0x100 - b3
                endif

            else
                set b3 = 0x80
                set b2 = 0x00
                set b1 = 0x00
                set b0 = 0x00
            endif

        else
            set b3 = 0x00
            set b2 = 0x00
            set b1 = 0x00
            set b0 = 0x00
        endif
endif

Yeah it is a lot more code but only in one place, but Binary is a lot less LOC so that is why I choose it.

Some nitpicking of Bitwise:
JASS:
        set number = 0x85ABCDEF

        set b3 = Bitwise.shiftr(Bitwise.AND32(number, 0xFF000000), 24)
        call BJDebugMsg(I2S(b3)) // -0x85

        set b3 = Bitwise.shiftr(Bitwise.AND32(number, 0xFF000000), 24)
        call BJDebugMsg(I2S(b3 + 0x100)) // +0x85

        set number = 0x710000AB
        call BJDebugMsg(I2S(Bitwise.shiftr(number, 29))) // 3, correct

        set number = 0xC10000AB
        call BJDebugMsg(I2S(Bitwise.shiftr(number, 30))) // 0?? expected: -1, or 3 =)

Anyway I update the script and the map with the new base64.write_int().
 
Hm.....

Why this over a more general library? What's important about using radix 64?


It seems like you're trying to do some sort of save/load thing? : o


Also, the Binary lib you are using can't handle 32 bits. It can only handle 31. Just making sure you know that. We do have some binary libs on the hive that can handle 32 bits : ).

You might have read this post and seen that Binary from wc3c is faster for 32 bits.

http://www.hiveworkshop.com/forums/jass-resources-412/snippet-bitwise-249223/

I say that this is only the case because Binary form wc3c doesn't actually handle 32 bits ; P.

wait, can't your Base library do this?
 
Level 23
Joined
Apr 16, 2012
Messages
4,041
Most definately cant, since base64 is actually not real base64, its called base64 because it uses 64 different characters, not 64 bits, + that resource is not available on hive, so I dont see why this couldn't take the spot.

I am for approval on this resource, but I will wait for another mod to voice his opinion over this.
 
Top