== Lesson 6: Lists of Things image::learn/06-tables.png[The finished lesson, 480] You have a hundred and twenty stars and not one of them exists. There is no star anywhere in your program: there is a loop that runs a hundred and twenty times, and each time round it works out two numbers from the star's number and its layer, plots a pixel, and forgets everything. The three layers are there to fake the one thing that arrangement cannot do, which is let a star be different from its neighbour. This lesson gives you the missing thing. You will learn how to keep a list of items, how to make one item that holds several named values at once, and then how to put the two together into a list of items that each remember their own position, speed, and brightness. It is the longest lesson in this part of the book and the one everything afterwards leans on. Take your time with it. === Start a New Folder A folder called `tables`, a file in it called `tables.singe`, and `Singe -R tables` running in a terminal. Copy lesson five's finished program into it: the star field, the ship, and all. === A List Add this line at the top of `tables.singe`, under the other values, and the three `overlayPrint` calls to `onOverlayUpdate` above the `return`. [source,lua] ---- local names = { "Ada", "Grace", "Alan" } ---- [source,lua] ---- overlayPrint(2, 2, names[1]) overlayPrint(2, 3, names[3]) overlayPrint(2, 4, "There are " .. #names .. " names.") ---- Run it. `Ada` on one line, `Alan` on the next, and `There are 3 names.` under them. The braces on the first line make a *table*. A table is the one container Lua has, and everything you will ever store more than one of goes in one. Here it is holding three strings in a row, which is the shape people call a *list*. The brackets on the next lines take an item out. `names[1]` is the first item, `names[3]` is the third, and the number in the brackets is called the *index*. That number is where Lua parts company with almost every other language you will meet. *Lua counts from one.* C, Java, Python, JavaScript, and most of the rest count from zero, so their first item is item zero and their third item is index two. Lua's first item is item one, and this is the single thing that trips up programmers arriving from somewhere else. You are arriving from nowhere, so you have the easier job: the first one is number one. `names[0]` is not an error, by the way. It is `nil` -- the nothing value from lesson three -- because there is nothing stored there. Reading an index that was never filled always gives `nil`, which is a kindness right up until you try to use it. The `#` in front of a table's name is the last new punctuation for a while. It hands you how many items are in the list, so `#names` is `3`. Because the count and the last index are the same number, `names[#names]` is always the last item, however long the list gets. === Adding and Taking Away A list you have to write out in full is not much use to a game. These two calls are how a list changes while the program runs. [source,lua] ---- table.insert(names, "Katherine") ---- That puts `Katherine` on the end, and `#names` is now `4`. `table.insert` is a function like any other, and the dot in the middle of its name means it lives inside a table called `table` that Lua provides -- a table of functions for working on tables. You do not have to make it or load it. It is there. Give `table.insert` three arguments and the middle one is where to put the new item. [source,lua] ---- table.insert(names, 1, "Edsger") ---- `Edsger` goes in at the front, and everything that was already there shuffles along to make room: `Ada` was item one and is now item two. Nothing is overwritten and nothing is lost. Taking an item out is the mirror image. [source,lua] ---- table.remove(names, 2) ---- Item two goes, and everything after it shuffles back down to close the gap, so the list never has a hole in the middle. `table.remove` also hands back the item it took out, so `local gone = table.remove(names, 2)` gets rid of it and tells you what it was in one line. Leave the number off altogether -- `table.remove(names)` -- and it takes the last item, which is the quickest way to use a list as a pile. Put those three calls in your script, one at a time, run it after each, and watch the count on the third line change. === Walking a List You know one way to visit every item, because you have been using it since lesson four: [source,lua] ---- for i = 1, #names do overlayPrint(2, i + 1, names[i]) end ---- That works. Start at one, stop at the count, use the counter as the index. There is a better way, and it is the way you will write from now on: [source,lua] ---- for i, name in ipairs(names) do overlayPrint(2, i + 1, name) end ---- `ipairs` walks a list from item one until it runs out, and hands your loop two things each time round: the index and the item itself. That second one is the point. With the numeric `for` you get a number and have to go back to the table with `names[i]` to find out what it is; with `ipairs` the item is already in your hand, under whatever name you put second. It is also harder to get wrong. `for i = 1, #names` is a place to make an off-by-one mistake -- start at nought and you print a `nil`, stop at `#names - 1` and you lose the last one -- and `ipairs` has no numbers in it at all to get wrong. Most of the time you do not want the index, only the item. Write the index as a single underscore and be done with it: [source,lua] ---- for _, name in ipairs(names) do ---- `_` is a perfectly ordinary name, and nothing in Lua treats it specially. It is a habit programmers share for saying "something arrives here and I am not going to use it". Anybody reading your loop knows at a glance that the index does not matter in it. Delete the `names` experiments now, and the `overlayPrint` lines with them. You know what a list is. Time to put something better than a string in one. === A Table Can Also Be a Record A star needs to remember four things: where it is across, where it is down, how fast it falls, and how bright it is. Four separate lists, all kept in step by hand, would be miserable. Instead, make a table and put names inside it. [source,lua] ---- local star = {} star.x = 40 star.y = 0 star.speed = 2 star.shade = 180 ---- The empty braces make an empty table. The four lines after it put values into it under names instead of numbers, and the dot is how you say which name you mean. `star.x` is read as "the `x` of `star`", and it behaves exactly like any other variable: read it, assign to it, add to it. [source,lua] ---- star.y = star.y + star.speed ---- A table used this way -- a fixed set of named values describing one thing -- is a *record*, and the names inside it are its *fields*. It is the same kind of table as the list above. The only difference is that a list's items are found by number and a record's fields are found by name, and one table can do both at once if you ever want it to. The dot is a shorthand. `star.x` and `star["x"]` mean exactly the same thing, which is worth seeing once: the brackets take a name as happily as they take a number, so a list and a record really are one idea wearing two hats. Use the dot. It reads better. A field you never set is `nil`, the same as an index you never filled. There is no list of allowed field names anywhere, and nothing checks your spelling, so `star.sped` is not an error -- it is `nil`, and you will meet what happens next at the end of this lesson. Writing four lines to fill in four fields gets old. You can put the fields inside the braces when you make the table: [source,lua] ---- local star = { x = 40, y = 0, speed = 2, shade = 180 } ---- Same table, one statement. The commas separate the fields, the last one needs no comma after it, and the lines are spread out only to be read easily. === A List of Records Here is the whole idea of this lesson in two lines: [source,lua] ---- local stars = {} table.insert(stars, star) ---- A list can hold anything, and that includes a table. So a list of records is a list of things, each of which remembers everything about itself. `stars[1]` is a whole star, and `stars[1].y` is that star's distance down the screen. Nearly every game you will ever write is a handful of lists like this one: the enemies, the bullets, the pickups, and the falling blocks. There is one thing about tables you must know before you write the code, and it catches everybody once. Handing a table to a function does not copy it. A number gets copied -- that is why `half` could chop up its `number` in lesson five without disturbing `drift` -- but a table is shared. Both names refer to the same table, and a change made through one is visible through the other. [source,lua] ---- local star = stars[3] star.y = star.y + 1 ---- That moves the third star in the list. Not a copy of it: it. This is exactly what you want here, and it is what makes the next section work, but remember it when a function you wrote changes something you did not expect it to. === Rebuild the Star Field Out goes the arithmetic. A star is about to become a thing. Delete `starAt`, `shadeFor`, `drawLayer`, and the old `drawStar`, and with them the constants they used: `LAYERS`, `STARS_PER_LAYER`, `ACROSS_STEP`, `DOWN_STEP`, `LAYER_SHIFT`, `LAYER_SPEED`, and `drift`. The `drift` line at the top of `onOverlayUpdate` goes as well. `LAYER_BRIGHT` stays, under a name that no longer mentions layers, and three values join it: [source,lua] ---- local STAR_COUNT = 120 local STAR_SHADE = 55 local BURST_COUNT = 10 ---- [source,lua] ---- local stars = {} ---- A hundred and twenty is what you had: three layers of forty. The list starts empty, and this is what fills it: [source,lua] ---- local function newStar(y) local speed = math.random(1, 3) local star = { x = math.random(0, width - 1), y = y, speed = speed, shade = math.random(60, 90) + speed * STAR_SHADE } return star end ---- `newStar` makes one star and hands it back. Look at the last field and compare it with `shadeFor` in lesson five: the same sum, with the star's own speed where the layer number used to be. The layers have not gone away so much as dissolved. Every star now picks its own speed, one, two, or three pixels a frame, and a star that falls faster is brighter, which is the whole of the illusion of depth and costs one line. The starting `y` is a parameter because the two callers want different things. The stars made when the game starts should be scattered all over the screen; the ones made later come in at the top. [source,lua] ---- local function makeStars() for number = 1, STAR_COUNT do table.insert(stars, newStar(math.random(0, height - 1))) end end ---- Nothing calls `makeStars` yet. Add one line at the very bottom of the file, after everything else and inside no function at all: [source,lua] ---- makeStars() ---- There is more to say about that line, and it waits until the end of the lesson. Now the drawing, which gets shorter every time you touch it: [source,lua] ---- local function drawStar(star) colorForeground(star.shade, star.shade, star.shade) overlayPlot(star.x, star.y) end local function drawStars() for _, star in ipairs(stars) do drawStar(star) end end ---- `drawStar` took three numbers in lesson five and takes one star now. Everything it needs to know is inside the star, which is the reason to have records at all: one thing to pass around instead of four, and no chance of handing them over in the wrong order. And `drawStars` has lost its loop over layers, along with any interest in how many stars there are or where they came from. === Stars That Come and Go Now the part the old star field could not do at all. Stars fall at their own speeds, leave at the bottom, and are replaced at the top. [source,lua] ---- local function moveStars() for i = #stars, 1, -1 do local star = stars[i] star.y = star.y + star.speed if star.y >= height then table.remove(stars, i) end end while #stars < STAR_COUNT do table.insert(stars, newStar(0)) end end ---- Call it from `onOverlayUpdate`, on the line where `drift = drift + 1` used to be. Each star moves down by its own `speed`, which is the line lesson five could not write. The `while` at the bottom is the refill: while there are fewer than a hundred and twenty stars, make another one at the top of the screen. It runs as many times as it needs to and then stops, which is the right tool when you do not know how many are missing. The `for` line at the top counts down, using the step you met in lesson four. `for i = #stars, 1, -1` visits the last item first and works back to item one. That is not a flourish. It is the only safe way to do what this loop does, and the next section is why. === Never Remove Going Forwards Write the same loop forwards and it is broken: [source,lua] ---- for i = 1, #stars do local star = stars[i] star.y = star.y + star.speed if star.y >= height then table.remove(stars, i) end end ---- Follow it by hand with four stars, and suppose stars two and three have both reached the bottom. * `i` is `1`. Star one is fine. * `i` is `2`. Star two is off the bottom, so out it goes. Everything shuffles down: the star that was three is now two, and the star that was four is now three. * `i` is `3`. That is the star that used to be four. *Star three was never looked at.* Removing item two moved it down into slot two, and the loop had already been to slot two and gone. * `i` is `4`. There is no item four any more. `stars[4]` is `nil`. Two separate faults from one small mistake. Items get skipped, so a star that had reached the bottom is not even looked at this time round. A star surviving one extra frame does no harm, and the same loop over a list of enemies, where being looked at is what kills them, is a bug you would chase for an hour. The second fault is louder. The list is now shorter than the count the loop started with, because `#stars` was worked out once, before any removal, so the last turns of the loop reach past the end of the list: ---- Error executing function 'onOverlayUpdate': tables.singe:60: attempt to index a nil value (local 'star') stack traceback: tables.singe:60: in upvalue 'moveStars' tables.singe:124: in function 'onOverlayUpdate' ---- "Attempt to index a nil value" means you put a dot after something that held nothing. The `(local 'star')` on the end names it: `star` was `nil`, so `star.y` had nowhere to look. Going backwards fixes both faults at once, and it is worth seeing why rather than taking it on trust. When you remove item `i`, everything after `i` moves down a slot -- and going backwards, everything after `i` is everything you have already visited. What is still to come, items `i - 1` down to `1`, has not moved at all. The list shrinking behind you cannot hurt you either, because you are walking towards item one and item one is always there. The rule, which you will use for the rest of your life: *when a loop might remove items from the list it is walking, walk it backwards.* `ipairs` is not an exception to this. Removing items in the middle of an `ipairs` loop goes wrong in the same way. `ipairs` is for looking, and the backwards `for` is for changing. === Add Some Yourself One more branch in `onInputPressed` and you can watch the list grow while the game runs. This goes with the other two: [source,lua] ---- elseif what == SWITCH_BUTTON1 then addBurst() ---- [source,lua] ---- local function addBurst() for number = 1, BURST_COUNT do table.insert(stars, newStar(0)) end end ---- `addBurst` goes above the callbacks with your other functions. And so that you can see the list's length change, put the count on screen in `onOverlayUpdate`: [source,lua] ---- overlayPrint(0, 0, "Stars: " .. #stars) ---- Press the button -- the space bar, unless you have changed what `SWITCH_BUTTON1` comes from -- a dozen times. The count goes up by ten each press and the sky thickens, and there is no repeat while you hold it down: `onInputPressed` is told once, when the button goes down. Then watch the count come back to a hundred and twenty on its own as the extra stars reach the bottom and are removed, because the refill only ever tops the list up to `STAR_COUNT` and never trims it. Nothing in the drawing, the moving, or the removing had to be told that the number of stars had changed. They all work from the list. === The Whole Thing [source,lua] ---- local STAR_COUNT = 120 local STAR_SHADE = 55 local BURST_COUNT = 10 local PLAYER_WIDTH = 20 local PLAYER_HEIGHT = 6 local PLAYER_SPEED = 3 local PLAYER_MARGIN = 6 local width = overlayGetWidth() local height = overlayGetHeight() local stars = {} local playerX = width // 2 local playerY = height - PLAYER_HEIGHT - PLAYER_MARGIN local movingLeft = false local movingRight = false local function clamp(value, low, high) if value < low then return low end if value > high then return high end return value end local function newStar(y) local speed = math.random(1, 3) local star = { x = math.random(0, width - 1), y = y, speed = speed, shade = math.random(60, 90) + speed * STAR_SHADE } return star end local function makeStars() for number = 1, STAR_COUNT do table.insert(stars, newStar(math.random(0, height - 1))) end end local function addBurst() for number = 1, BURST_COUNT do table.insert(stars, newStar(0)) end end local function moveStars() for i = #stars, 1, -1 do local star = stars[i] star.y = star.y + star.speed if star.y >= height then table.remove(stars, i) end end while #stars < STAR_COUNT do table.insert(stars, newStar(0)) end end local function drawStar(star) colorForeground(star.shade, star.shade, star.shade) overlayPlot(star.x, star.y) end local function drawStars() for _, star in ipairs(stars) do drawStar(star) end end local function drawPlayer() colorForeground(80, 255, 120) overlayBox(playerX, playerY, playerX + PLAYER_WIDTH, playerY + PLAYER_HEIGHT) end local function movePlayer() if movingLeft then playerX = playerX - PLAYER_SPEED end if movingRight then playerX = playerX + PLAYER_SPEED end playerX = clamp(playerX, 0, width - PLAYER_WIDTH - 1) end function onInputPressed(what) if what == SWITCH_LEFT then movingLeft = true elseif what == SWITCH_RIGHT then movingRight = true elseif what == SWITCH_BUTTON1 then addBurst() end end function onInputReleased(what) if what == SWITCH_LEFT then movingLeft = false elseif what == SWITCH_RIGHT then movingRight = false end end function onOverlayUpdate() overlayClear() movePlayer() moveStars() drawStars() drawPlayer() overlayPrint(0, 0, "Stars: " .. #stars) return OVERLAY_UPDATED end makeStars() ---- === What Just Happened The program is shorter than lesson five's and it does something lesson five could not begin to do. Three pieces are worth a second look. [source,lua] ---- makeStars() ---- That single line at the very bottom of the file is not inside any function, and it is easy to miss. Lua reads your file from top to bottom when Singe loads it, and a line at the outer level of the file runs there and then, once, before the engine has called anything. Everything above it is definitions -- here is a value, here is a function -- and this is the one instruction that happens at load time. It has to be at the bottom, because `makeStars` does not exist until the line that creates it has been read. [source,lua] ---- local star = stars[i] star.y = star.y + star.speed ---- `star` is a local made fresh on every turn of the loop, and it holds the same table that is sitting in the list -- not a copy of it. Changing `star.y` moves the star that is in the list. If tables were copied, this loop would move a hundred and twenty copies a frame and nothing on screen would ever change. [source,lua] ---- for _, star in ipairs(stars) do drawStar(star) end ---- Compare that with lesson four: two nested loops, two pieces of arithmetic per star, a remainder to keep each one on the screen, and three layers to fake the variety you now get for nothing. The drawing no longer knows or cares how the stars got where they are, how many there are, or what happens to them next. Each part of the program does one job, and the list is what they all agree about. === Try It . *Make it deeper.* Change `math.random(1, 3)` to `math.random(1, 6)` and `STAR_SHADE` to `25`. More speeds, more shades, a deeper sky. . *Fewer stars, bigger burst.* Set `STAR_COUNT` to `10` and `BURST_COUNT` to `100`, then press the button three or four times. Watch the count on screen climb, and time how long it takes to drain. . *Give a star another field.* Add `age = 0` to `newStar`, add one to `star.age` in `moveStars`, and put `stars[1].age` on screen next to the count. Nothing else in the program needs changing to carry a new fact about every star. . *Stop removing them.* Delete the three lines of the `if star.y >= height` block from `moveStars` and run it. Wait ten seconds, then read the count on screen and look at the sky. Work out how both of those can be true at once. . *Do it forwards.* Change `for i = #stars, 1, -1` to `for i = 1, #stars` and run it until it falls over. Then read the section above again with the real error in front of you. === Break It on Purpose Misspell a field name. In `moveStars`, change `star.speed` to `star.sped`: [source,lua] ---- star.y = star.y + star.sped ---- Run it: ---- Error executing function 'onOverlayUpdate': tables.singe:60: attempt to perform arithmetic on a nil value (field 'sped') stack traceback: tables.singe:60: in upvalue 'moveStars' tables.singe:124: in function 'onOverlayUpdate' ---- Your line numbers will not be exactly these unless your file matches the one above line for line. The name of the file, the number after it, and the complaint are what you read. Read the end of the first line first. `field 'sped'` is Lua telling you which name came up empty, and it is telling you it was a field -- something after a dot -- rather than a variable of your own. "Attempt to perform arithmetic on a nil value" is what it was doing when it found out: adding `nil` to a number, which cannot be done. There was no complaint when the table was made without a `sped` field, and none at the moment you asked for it. A table hands out `nil` for any name it does not have, without comment. The error comes later, at the first line that tries to *use* the nothing you were given, which may be a long way from the typo. When you see `nil value (field 'something')`, look for the place that name was spelled differently. === What You Learned * Braces make a table, the one container Lua has. A table used as a list holds items in a row, and brackets with an index take one out. * Lua counts from one. Most other languages count from zero. * `#list` is how many items are in it, and `list[#list]` is the last one. * `table.insert` adds an item, on the end or at a position you name; `table.remove` takes one out and closes the gap. * An index or a field that was never filled is `nil`, and nothing warns you until you use it. * `ipairs` walks a list and hands you the index and the item; `_` is the habit for an index you do not want. * A table used as a record holds named fields, reached with a dot. * A list of records is how a game holds its enemies, its bullets, and everything else there is more than one of. * Tables are shared, not copied, when you pass them around. * When a loop may remove items from the list it is walking, walk it backwards. === Next Time You have every piece of a game now: things that move, a ship that answers the controls, decisions, loops, functions, and a list that things can join and leave while the game is running. Lesson seven puts them together into a game you can lose. You steer along the bottom of the screen, blocks fall from the top, and you stay out of their way -- and the falling blocks are a list of records, made, moved, and removed exactly as your stars are.