singe/docs/lessons/06-tables.adoc

== Lesson 6: Lists of Things

image::learn/06-tables.png[The finished lesson, 480]

You have a hundred and twenty stars and not one of them exists. There is no
star anywhere in your program: there is a loop that runs a hundred and twenty
times, and each time round it works out two numbers from the star's number and
its layer, plots a pixel, and forgets everything. The three layers are there
to fake the one thing that arrangement cannot do, which is let a star be
different from its neighbour.

This lesson gives you the missing thing. You will learn how to keep a list of
items, how to make one item that holds several named values at once, and then
how to put the two together into a list of items that each remember their own
position, speed, and brightness. It is the longest lesson in this part of the
book and the one everything afterwards leans on. Take your time with it.

=== Start a New Folder

A folder called `tables`, a file in it called `tables.singe`, and
`Singe -R tables` running in a terminal. Copy lesson five's finished
program into it: the star field, the ship, and all.

=== A List

Add this line at the top of `tables.singe`, under the other values, and the
three `overlayPrint` calls to `onOverlayUpdate` above the `return`.

[source,lua]
----
local names = { "Ada", "Grace", "Alan" }
----

[source,lua]
----
    overlayPrint(2, 2, names[1])
    overlayPrint(2, 3, names[3])
    overlayPrint(2, 4, "There are " .. #names .. " names.")
----

Run it. `Ada` on one line, `Alan` on the next, and `There are 3 names.` under
them.

The braces on the first line make a *table*. A table is the one container Lua
has, and everything you will ever store more than one of goes in one. Here it
is holding three strings in a row, which is the shape people call a *list*.

The brackets on the next lines take an item out. `names[1]` is the first item,
`names[3]` is the third, and the number in the brackets is called the *index*.

That number is where Lua parts company with almost every other language you
will meet. *Lua counts from one.* C, Java, Python, JavaScript, and most of the
rest count from zero, so their first item is item zero and their third item is
index two. Lua's first item is item one, and this is the single thing that
trips up programmers arriving from somewhere else. You are arriving from
nowhere, so you have the easier job: the first one is number one.

`names[0]` is not an error, by the way. It is `nil` -- the nothing value from
lesson three -- because there is nothing stored there. Reading an index that
was never filled always gives `nil`, which is a kindness right up until you
try to use it.

The `#` in front of a table's name is the last new punctuation for a while. It
hands you how many items are in the list, so `#names` is `3`. Because the
count and the last index are the same number, `names[#names]` is always the
last item, however long the list gets.

=== Adding and Taking Away

A list you have to write out in full is not much use to a game. These two
calls are how a list changes while the program runs.

[source,lua]
----
    table.insert(names, "Katherine")
----

That puts `Katherine` on the end, and `#names` is now `4`. `table.insert` is a
function like any other, and the dot in the middle of its name means it lives
inside a table called `table` that Lua provides -- a table of functions for
working on tables. You do not have to make it or load it. It is there.

Give `table.insert` three arguments and the middle one is where to put the new
item.

[source,lua]
----
    table.insert(names, 1, "Edsger")
----

`Edsger` goes in at the front, and everything that was already there shuffles
along to make room: `Ada` was item one and is now item two. Nothing is
overwritten and nothing is lost.

Taking an item out is the mirror image.

[source,lua]
----
    table.remove(names, 2)
----

Item two goes, and everything after it shuffles back down to close the gap, so
the list never has a hole in the middle. `table.remove` also hands back the
item it took out, so `local gone = table.remove(names, 2)` gets rid of it and
tells you what it was in one line. Leave the number off altogether --
`table.remove(names)` -- and it takes the last item, which is the quickest way
to use a list as a pile.

Put those three calls in your script, one at a time, run it after each, and
watch the count on the third line change.

=== Walking a List

You know one way to visit every item, because you have been using it since
lesson four:

[source,lua]
----
    for i = 1, #names do
        overlayPrint(2, i + 1, names[i])
    end
----

That works. Start at one, stop at the count, use the counter as the index.
There is a better way, and it is the way you will write from now on:

[source,lua]
----
    for i, name in ipairs(names) do
        overlayPrint(2, i + 1, name)
    end
----

`ipairs` walks a list from item one until it runs out, and hands your loop two
things each time round: the index and the item itself. That second one is the
point. With the numeric `for` you get a number and have to go back to the
table with `names[i]` to find out what it is; with `ipairs` the item is
already in your hand, under whatever name you put second.

It is also harder to get wrong. `for i = 1, #names` is a place to make an
off-by-one mistake -- start at nought and you print a `nil`, stop at
`#names - 1` and you lose the last one -- and `ipairs` has no numbers in it at
all to get wrong.

Most of the time you do not want the index, only the item. Write the index as
a single underscore and be done with it:

[source,lua]
----
    for _, name in ipairs(names) do
----

`_` is a perfectly ordinary name, and nothing in Lua treats it specially. It
is a habit programmers share for saying "something arrives here and I am not
going to use it". Anybody reading your loop knows at a glance that the index
does not matter in it.

Delete the `names` experiments now, and the `overlayPrint` lines with them.
You know what a list is. Time to put something better than a string in one.

=== A Table Can Also Be a Record

A star needs to remember four things: where it is across, where it is down,
how fast it falls, and how bright it is. Four separate lists, all kept in step
by hand, would be miserable. Instead, make a table and put names inside it.

[source,lua]
----
    local star = {}
    star.x = 40
    star.y = 0
    star.speed = 2
    star.shade = 180
----

The empty braces make an empty table. The four lines after it put values into
it under names instead of numbers, and the dot is how you say which name you
mean. `star.x` is read as "the `x` of `star`", and it behaves exactly like any
other variable: read it, assign to it, add to it.

[source,lua]
----
    star.y = star.y + star.speed
----

A table used this way -- a fixed set of named values describing one thing --
is a *record*, and the names inside it are its *fields*. It is the same kind
of table as the list above. The only difference is that a list's items are
found by number and a record's fields are found by name, and one table can do
both at once if you ever want it to.

The dot is a shorthand. `star.x` and `star["x"]` mean exactly the same thing,
which is worth seeing once: the brackets take a name as happily as they take a
number, so a list and a record really are one idea wearing two hats. Use the
dot. It reads better.

A field you never set is `nil`, the same as an index you never filled. There
is no list of allowed field names anywhere, and nothing checks your spelling,
so `star.sped` is not an error -- it is `nil`, and you will meet what happens
next at the end of this lesson.

Writing four lines to fill in four fields gets old. You can put the fields
inside the braces when you make the table:

[source,lua]
----
    local star = {
        x     = 40,
        y     = 0,
        speed = 2,
        shade = 180
    }
----

Same table, one statement. The commas separate the fields, the last one needs
no comma after it, and the lines are spread out only to be read easily.

=== A List of Records

Here is the whole idea of this lesson in two lines:

[source,lua]
----
    local stars = {}
    table.insert(stars, star)
----

A list can hold anything, and that includes a table. So a list of records is a
list of things, each of which remembers everything about itself. `stars[1]` is
a whole star, and `stars[1].y` is that star's distance down the screen. Nearly
every game you will ever write is a handful of lists like this one: the
enemies, the bullets, the pickups, and the falling blocks.

There is one thing about tables you must know before you write the code, and
it catches everybody once. Handing a table to a function does not copy it. A
number gets copied -- that is why `half` could chop up its `number` in lesson
five without disturbing `drift` -- but a table is shared. Both names refer to
the same table, and a change made through one is visible through the other.

[source,lua]
----
    local star = stars[3]
    star.y = star.y + 1
----

That moves the third star in the list. Not a copy of it: it. This is exactly
what you want here, and it is what makes the next section work, but remember
it when a function you wrote changes something you did not expect it to.

=== Rebuild the Star Field

Out goes the arithmetic. A star is about to become a thing.

Delete `starAt`, `shadeFor`, `drawLayer`, and the old `drawStar`, and with
them the constants they used: `LAYERS`, `STARS_PER_LAYER`, `ACROSS_STEP`,
`DOWN_STEP`, `LAYER_SHIFT`, `LAYER_SPEED`, and `drift`. The `drift` line at
the top of `onOverlayUpdate` goes as well. `LAYER_BRIGHT` stays, under a name
that no longer mentions layers, and three values join it:

[source,lua]
----
local STAR_COUNT  = 120
local STAR_SHADE  = 55
local BURST_COUNT = 10
----

[source,lua]
----
local stars = {}
----

A hundred and twenty is what you had: three layers of forty. The list starts
empty, and this is what fills it:

[source,lua]
----
local function newStar(y)
    local speed = math.random(1, 3)
    local star  = {
        x     = math.random(0, width - 1),
        y     = y,
        speed = speed,
        shade = math.random(60, 90) + speed * STAR_SHADE
    }
    return star
end
----

`newStar` makes one star and hands it back. Look at the last field and compare
it with `shadeFor` in lesson five: the same sum, with the star's own speed
where the layer number used to be. The layers have not gone away so much as
dissolved. Every star now picks its own speed, one, two, or three pixels a
frame, and a star that falls faster is brighter, which is the whole of the
illusion of depth and costs one line.

The starting `y` is a parameter because the two callers want different things.
The stars made when the game starts should be scattered all over the screen;
the ones made later come in at the top.

[source,lua]
----
local function makeStars()
    for number = 1, STAR_COUNT do
        table.insert(stars, newStar(math.random(0, height - 1)))
    end
end
----

Nothing calls `makeStars` yet. Add one line at the very bottom of the file,
after everything else and inside no function at all:

[source,lua]
----
makeStars()
----

There is more to say about that line, and it waits until the end of the
lesson. Now the drawing, which gets shorter every time you touch it:

[source,lua]
----
local function drawStar(star)
    colorForeground(star.shade, star.shade, star.shade)
    overlayPlot(star.x, star.y)
end


local function drawStars()
    for _, star in ipairs(stars) do
        drawStar(star)
    end
end
----

`drawStar` took three numbers in lesson five and takes one star now. Everything
it needs to know is inside the star, which is the reason to have records at
all: one thing to pass around instead of four, and no chance of handing them
over in the wrong order. And `drawStars` has lost its loop over layers, along
with any interest in how many stars there are or where they came from.

=== Stars That Come and Go

Now the part the old star field could not do at all. Stars fall at their own
speeds, leave at the bottom, and are replaced at the top.

[source,lua]
----
local function moveStars()
    for i = #stars, 1, -1 do
        local star = stars[i]
        star.y = star.y + star.speed
        if star.y >= height then
            table.remove(stars, i)
        end
    end
    while #stars < STAR_COUNT do
        table.insert(stars, newStar(0))
    end
end
----

Call it from `onOverlayUpdate`, on the line where `drift = drift + 1` used to
be.

Each star moves down by its own `speed`, which is the line lesson five could
not write. The `while` at the bottom is the refill: while there are fewer than
a hundred and twenty stars, make another one at the top of the screen. It runs
as many times as it needs to and then stops, which is the right tool when you
do not know how many are missing.

The `for` line at the top counts down, using the step you met in lesson four.
`for i = #stars, 1, -1` visits the last item first and works back to item one.
That is not a flourish. It is the only safe way to do what this loop does, and
the next section is why.

=== Never Remove Going Forwards

Write the same loop forwards and it is broken:

[source,lua]
----
    for i = 1, #stars do
        local star = stars[i]
        star.y = star.y + star.speed
        if star.y >= height then
            table.remove(stars, i)
        end
    end
----

Follow it by hand with four stars, and suppose stars two and three have both
reached the bottom.

* `i` is `1`. Star one is fine.
* `i` is `2`. Star two is off the bottom, so out it goes. Everything shuffles
  down: the star that was three is now two, and the star that was four is now
  three.
* `i` is `3`. That is the star that used to be four. *Star three was never
  looked at.* Removing item two moved it down into slot two, and the loop had
  already been to slot two and gone.
* `i` is `4`. There is no item four any more. `stars[4]` is `nil`.

Two separate faults from one small mistake. Items get skipped, so a star that
had reached the bottom is not even looked at this time round. A star surviving
one extra frame does no harm, and the same loop over a list of enemies, where
being looked at is what kills them, is a bug you would chase for an hour. The
second fault is louder. The list is now shorter than the count the loop
started with, because `#stars` was worked out once, before any removal, so the
last turns of the loop reach past the end of the list:

----
Error executing function 'onOverlayUpdate':  tables.singe:60: attempt to index a nil value (local 'star')
stack traceback:
	tables.singe:60: in upvalue 'moveStars'
	tables.singe:124: in function 'onOverlayUpdate'
----

"Attempt to index a nil value" means you put a dot after something that held
nothing. The `(local 'star')` on the end names it: `star` was `nil`, so
`star.y` had nowhere to look.

Going backwards fixes both faults at once, and it is worth seeing why rather
than taking it on trust. When you remove item `i`, everything after `i` moves
down a slot -- and going backwards, everything after `i` is everything you
have already visited. What is still to come, items `i - 1` down to `1`, has
not moved at all. The list shrinking behind you cannot hurt you either,
because you are walking towards item one and item one is always there.

The rule, which you will use for the rest of your life: *when a loop might
remove items from the list it is walking, walk it backwards.*

`ipairs` is not an exception to this. Removing items in the middle of an
`ipairs` loop goes wrong in the same way. `ipairs` is for looking, and the
backwards `for` is for changing.

=== Add Some Yourself

One more branch in `onInputPressed` and you can watch the list grow while the
game runs. This goes with the other two:

[source,lua]
----
    elseif what == SWITCH_BUTTON1 then
        addBurst()
----

[source,lua]
----
local function addBurst()
    for number = 1, BURST_COUNT do
        table.insert(stars, newStar(0))
    end
end
----

`addBurst` goes above the callbacks with your other functions. And so that you
can see the list's length change, put the count on screen in
`onOverlayUpdate`:

[source,lua]
----
    overlayPrint(0, 0, "Stars: " .. #stars)
----

Press the button -- the space bar, unless you have changed what
`SWITCH_BUTTON1` comes from -- a dozen times. The count goes up by ten each
press and the sky thickens, and there is no repeat while you hold it down:
`onInputPressed` is told once, when the button goes down. Then watch the count
come back to a hundred and twenty on its own as the extra stars reach the
bottom and are removed, because the refill only ever tops the list up to
`STAR_COUNT` and never trims it.

Nothing in the drawing, the moving, or the removing had to be told that the
number of stars had changed. They all work from the list.

=== The Whole Thing

[source,lua]
----
local STAR_COUNT  = 120
local STAR_SHADE  = 55
local BURST_COUNT = 10

local PLAYER_WIDTH  = 20
local PLAYER_HEIGHT = 6
local PLAYER_SPEED  = 3
local PLAYER_MARGIN = 6

local width  = overlayGetWidth()
local height = overlayGetHeight()

local stars       = {}
local playerX     = width // 2
local playerY     = height - PLAYER_HEIGHT - PLAYER_MARGIN
local movingLeft  = false
local movingRight = false


local function clamp(value, low, high)
    if value < low then
        return low
    end
    if value > high then
        return high
    end
    return value
end


local function newStar(y)
    local speed = math.random(1, 3)
    local star  = {
        x     = math.random(0, width - 1),
        y     = y,
        speed = speed,
        shade = math.random(60, 90) + speed * STAR_SHADE
    }
    return star
end


local function makeStars()
    for number = 1, STAR_COUNT do
        table.insert(stars, newStar(math.random(0, height - 1)))
    end
end


local function addBurst()
    for number = 1, BURST_COUNT do
        table.insert(stars, newStar(0))
    end
end


local function moveStars()
    for i = #stars, 1, -1 do
        local star = stars[i]
        star.y = star.y + star.speed
        if star.y >= height then
            table.remove(stars, i)
        end
    end
    while #stars < STAR_COUNT do
        table.insert(stars, newStar(0))
    end
end


local function drawStar(star)
    colorForeground(star.shade, star.shade, star.shade)
    overlayPlot(star.x, star.y)
end


local function drawStars()
    for _, star in ipairs(stars) do
        drawStar(star)
    end
end


local function drawPlayer()
    colorForeground(80, 255, 120)
    overlayBox(playerX, playerY, playerX + PLAYER_WIDTH, playerY + PLAYER_HEIGHT)
end


local function movePlayer()
    if movingLeft then
        playerX = playerX - PLAYER_SPEED
    end
    if movingRight then
        playerX = playerX + PLAYER_SPEED
    end
    playerX = clamp(playerX, 0, width - PLAYER_WIDTH - 1)
end


function onInputPressed(what)
    if what == SWITCH_LEFT then
        movingLeft = true
    elseif what == SWITCH_RIGHT then
        movingRight = true
    elseif what == SWITCH_BUTTON1 then
        addBurst()
    end
end


function onInputReleased(what)
    if what == SWITCH_LEFT then
        movingLeft = false
    elseif what == SWITCH_RIGHT then
        movingRight = false
    end
end


function onOverlayUpdate()
    overlayClear()
    movePlayer()
    moveStars()
    drawStars()
    drawPlayer()
    overlayPrint(0, 0, "Stars: " .. #stars)
    return OVERLAY_UPDATED
end


makeStars()
----

=== What Just Happened

The program is shorter than lesson five's and it does something lesson five
could not begin to do. Three pieces are worth a second look.

[source,lua]
----
makeStars()
----

That single line at the very bottom of the file is not inside any function,
and it is easy to miss. Lua reads your file from top to bottom when Singe
loads it, and a line at the outer level of the file runs there and then, once,
before the engine has called anything. Everything above it is definitions --
here is a value, here is a function -- and this is the one instruction that
happens at load time. It has to be at the bottom, because `makeStars` does not
exist until the line that creates it has been read.

[source,lua]
----
        local star = stars[i]
        star.y = star.y + star.speed
----

`star` is a local made fresh on every turn of the loop, and it holds the same
table that is sitting in the list -- not a copy of it. Changing `star.y` moves
the star that is in the list. If tables were copied, this loop would move a
hundred and twenty copies a frame and nothing on screen would ever change.

[source,lua]
----
    for _, star in ipairs(stars) do
        drawStar(star)
    end
----

Compare that with lesson four: two nested loops, two pieces of arithmetic per
star, a remainder to keep each one on the screen, and three layers to fake the
variety you now get for nothing. The drawing no longer knows or cares how the
stars got where they are, how many there are, or what happens to them next.
Each part of the program does one job, and the list is what they all agree
about.

=== Try It

. *Make it deeper.* Change `math.random(1, 3)` to `math.random(1, 6)` and
  `STAR_SHADE` to `25`. More speeds, more shades, a deeper sky.
. *Fewer stars, bigger burst.* Set `STAR_COUNT` to `10` and `BURST_COUNT` to
  `100`, then press the button three or four times. Watch the count on screen
  climb, and time how long it takes to drain.
. *Give a star another field.* Add `age = 0` to `newStar`, add one to
  `star.age` in `moveStars`, and put `stars[1].age` on screen next to the
  count. Nothing else in the program needs changing to carry a new fact about
  every star.
. *Stop removing them.* Delete the three lines of the `if star.y >= height`
  block from `moveStars` and run it. Wait ten seconds, then read the count on
  screen and look at the sky. Work out how both of those can be true at once.
. *Do it forwards.* Change `for i = #stars, 1, -1` to `for i = 1, #stars` and
  run it until it falls over. Then read the section above again with the real
  error in front of you.

=== Break It on Purpose

Misspell a field name. In `moveStars`, change `star.speed` to `star.sped`:

[source,lua]
----
        star.y = star.y + star.sped
----

Run it:

----
Error executing function 'onOverlayUpdate':  tables.singe:60: attempt to perform arithmetic on a nil value (field 'sped')
stack traceback:
	tables.singe:60: in upvalue 'moveStars'
	tables.singe:124: in function 'onOverlayUpdate'
----

Your line numbers will not be exactly these unless your file matches the one
above line for line. The name of the file, the number after it, and the
complaint are what you read.

Read the end of the first line first. `field 'sped'` is Lua telling you which
name came up empty, and it is telling you it was a field -- something after a
dot -- rather than a variable of your own. "Attempt to perform arithmetic on a
nil value" is what it was doing when it found out: adding `nil` to a number,
which cannot be done.

There was no complaint when the table was made without a `sped` field, and
none at the moment you asked for it. A table hands out `nil` for any name it
does not have, without comment. The error comes later, at the first line that
tries to *use* the nothing you were given, which may be a long way from the
typo. When you see `nil value (field 'something')`, look for the place that
name was spelled differently.

=== What You Learned

* Braces make a table, the one container Lua has. A table used as a list holds
  items in a row, and brackets with an index take one out.
* Lua counts from one. Most other languages count from zero.
* `#list` is how many items are in it, and `list[#list]` is the last one.
* `table.insert` adds an item, on the end or at a position you name;
  `table.remove` takes one out and closes the gap.
* An index or a field that was never filled is `nil`, and nothing warns you
  until you use it.
* `ipairs` walks a list and hands you the index and the item; `_` is the habit
  for an index you do not want.
* A table used as a record holds named fields, reached with a dot.
* A list of records is how a game holds its enemies, its bullets, and
  everything else there is more than one of.
* Tables are shared, not copied, when you pass them around.
* When a loop may remove items from the list it is walking, walk it backwards.

=== Next Time

You have every piece of a game now: things that move, a ship that answers the
controls, decisions, loops, functions, and a list that things can join and
leave while the game is running.

Lesson seven puts them together into a game you can lose. You steer along the
bottom of the screen, blocks fall from the top, and you stay out of their way
-- and the falling blocks are a list of records, made, moved, and removed
exactly as your stars are.