Annoying quirk with lua '#' length operator: finding first non-nil value in a straight array

[pr114]

Just wanted to mention something about the # operator that was not working as expected. I use this for my systems to efficiently manage data in the map. I DON'T want to have to iterate over (long) lists each time.

Now what I expected was that # returns the last not-nil value in a straight array.
But unfortunately, it doesn't work with setting values to nil. Like it's not updating its internal statistic.

Example:

l = {11,22,33,44,55}
l[3] = nil
print(#l) >returns 5

Somehow, nilling a certain value ends up updating the statistic. So if you do something like this:

l = {11,22,33,44,55}
l[3] = nil
x = l[5]
l[5] = nil
print(#l) > statistics updated: now returns 2
l[5] = x > (returning to prior value)

I'm a little bit stuck here, because if the table is not creating like l = {11,22,33,44,55}, but instead like this:
l = {}
for x=1,5 do l[x]=x*11 end
Now suddenly not nilling element 5, but element 4 updates the statistics.

Meanwhile I'm experimenting a bit, but could someone with insight in lua chime in, and help me find the most efficient way to find a non-nil value in a straight array

 

[Drake53]

Have you tried using table.remove?

 

[pr114]

Actually, in the case the order does not matter table.remove will be a good option.
I have a system that involves items in inventory, but this is so small that I might as well iterate over it.

I'm finding on the net that basically # is just not suited for finding the first non-nil value.

 

[Aniki]

I think in Lua when you set a table/map's key to 'nil' it's equivalent to removing that key-value pair (similarly to JavaScript's "delete map[key]"). The manual's entry about the '#' operator (the length operator) explains how it relates to nil values in a table (returning a random "border"). If you can't/don't want to use 'table.remove' then you can make your own nil/sentinel value and search for that instead:

local mynil = {}
local t = {11, 22, 33, 44, 55}

local function findIdxFirstMyNil(t)
    -- there's no linear search builtin in Lua (as far as I know)
    for idx, v in ipairs(t) do
        if v == mynil then
            return idx
        end
    end
    return 0 -- no 'mynil's
end

print(findIdxFirstMyNil(t)) -- 0
print(#t) -- 5

t[3] = mynil
print(findIdxFirstMyNil(t)) -- 3
print(#t) -- 5
[/lua]

 

[pr114]

Yeah, I've done a bit of searching on this particular topic, and it turns out that the # operator looks for holes in a sequence by using a binary search. That explains its efficiency, but also why some holes do get picked up, and others don't. Though # is reliable with a straight sequence.

To have more performance in my map I have decided against iterating the entire table. It is concerning a unit table that can have a length of up to 250 units. I know lua is generally very quick, but it didn't feel well with me to use this.

I remade the system to use table.remove and keeping track of a few factors with count variables. I haven't benchmarked this to compare to a iterating method, but I think table.remove will surely perform better on long lists than iterating.

 

[Prometheus3375]

I think in Lua when you set a table/map's key to 'nil' it's equivalent to removing that key-value pair

+