Four-corner collision detection

Say you want to calculate overlap between an object and a rectangular grid of cells that may be solid or not solid. Assume you know the position of an object's center (in world space, relative to the grid's origin), its radius (distance from the center to the edge of the bounding box) in X and Y directions. Also assume at first that the radius is less than half the length of a grid square, that is, the object is no larger than a grid cell.

This algorithm detects whether the block is penetrating a solid cell and which way it should be pushed so that it is no longer inside a solid cell. It has been prototyped in Python as part of Wrecking Ball Boy.

Background
Numerous two-dimensional games for the NES, Super NES, and other 2D platforms make their maps out of a rectangular grid of tiles. Each cell in the grid can be solid or empty. A solid cell, for example, is one that an object is never supposed to enter, such as a wall or floor. Some cells may be directionally solid, in that they are solid if an object enters from one direction but allow passing through in the opposite direction, such as most platforms in World 1-1 of Super Mario Bros. 2. Some cells may be solid in one part and empty in another part, such as slopes.

A common method of detecting collisions involved sampling the grid at various points relative to a game object's position. A widely cited article about the implementation of a platform game on a 1980s 8-bit machine, "Programming M.C. Kids", discusses how the game detects collisions by sampling the grid at eight points around an object's perimeter, with some points not checked depending on the direction of motion. And once the program detects a collision, it must respond to the collision by ejecting the object and possibly changing its velocity or movement state.

Several games are known for glitchy collision detection and response in edge cases. Super Mario Bros. briefly puts Mario into the standing state when he enters a wall, allowing for wall jumping glitches, and the popular method of entering its World 36-1 relies on a flaw in its wall ejection. Tool-assisted speedruns of Mega Man series are known for exploiting bugs in the wall ejection behavior. More than once, I've got Sonic stuck inside a ramp in Sonic the Hedgehog 2 for Sega Genesis. And some games move objects around within the background, meaning one cell changes from solid to empty while another changes from empty to solid, which may confuse some of the game logic. I guess some of the poor collision response in some third- and fourth-generation games is due to a lack of rigorous reasoning about where to sample and what to do in each case.

This lack of rigor is what I aim to correct with the following algorithm that detects whether an object overlaps a solid cell and, if so, the closest position that does not overlap a solid cell.

Contour shape
This step characterizes the contour of the tiles that make up the neighborhood around the object.

The first thing to do is round the object's position to the nearest intersection of grid cells and calculate the displacement (dx, dy) from the intersection to the object's center.

Then sample the solidity of the cells at the four corners of the object's bounding box. Sampling the solidity involves rounding each corner to the nearest cell center and retrieving the tile there.

On some retro platforms, it may be faster to round the position once and then retrieve other neighboring cells relative to the first cell's position. This is also true if the object's bounding box is entirely embedded in a solid cell, as an empty tile may be nearby toward which it can be pushed. In this case, you can retrieve the solidity of the four cells surrounding that intersection. Then if the object's center is not currently embedded in a solid cell, and the absolute value of a displacement coordinate is greater than or equal to the radius, then the bounding box is completely in one of the squares, not straddling a grid line. Copy the column to the other column if dy <= -r or dy >= r, or copy the row to the other row if dx <= -r or dx >= r. After this, only those edges that the bounding box straddles will be included in the contour.

To handle directionally solid tiles, the engine must store the previous position of the object accurate to at least one tile. Assume without loss of generality that a particular tile type cannot be passed through downward. Treat such a tile as solid only if it's on the bottom row of the neighborhood and the previous position was completely above that tile.

Assign each solid cell a place value: 1 for top left, 2 for top right, 4 for bottom left, and 8 for bottom right, and combine these using bitwise OR (or equivalently addition). This associates each of 16 possible neighborhoods with a hexadecimal nibble:



In this representation, copying a row or column can be done with an AND, a shift of one of two bits, and an OR.

Edge collision
At this point, there are six general contour shapes, corresponding to the six distinct blocks in the video game Lumines or the six cases of the marching squares algorithm that are distinct up to rotation. The first step is to reduce the number of cases by expanding any isolated solid cells to the full width of the whole block.
 * 0
 * No collision. Return.


 * 1 ▘, 2 ▝, 4 ▖, 8 ▗
 * If the object is in the opposite corner from the solid cell, follow the "Corner collision" instructions below. Otherwise, construct a diagonal line from the solid cell to the opposite corner, find on which side of the line the object's center lies, fill in the block on the far corner, and follow the instructions for 3, 5, A, or C.


 * 3 ▀, 5 ▌, A ▐, C ▄
 * Push straight away from the wall, based on the difference between the object's radius and the current displacement.


 * 6 ▞, 9 ▚
 * Construct a diagonal line through both solid cells, find on which side of the line the center lies, fill in the block on the far corner, and follow the instructions for 7, B, D, or E.


 * 7 ▛, B ▜, D ▙, E ▟
 * This contains one of 3 or C and one of 5 or A. Process each wall.


 * F █
 * The object is embedded near the center. Find which of the four sides is closest to the last position that the object had while not overlapping. If that position is expensive to store, such as on an 8-bit platform with 2048 bytes of RAM, find which of the four sides is closest to the edge and push it one cell toward that side. By moving it exactly one cell per game cycle, you run a risk for a glitch like "zipping" in Mega Man, but at least the object is kicked out quickly.

Corner collision
The object is colliding with the corner of a solid tile. First determine whether the corner is actually within the object. For a circular object, evaluate the inequality dx2 + dy2 < r2, either through multiplication or through lookup tables of y = x2. If not, reject the collision. Otherwise, use dx and dy to determine how far to try pushing the object out of the corner.

Extensions
Extension to slopes and to objects larger than a cell is left as an exercise for the reader. For large objects, it involves sampling the object's perimeter. This may fail if a solid tile is completely embedded in an object.