Public_Skunkworks/singe
Public_Skunkworks/singe
1
0
Fork 0
Code Issues 7 Pull requests Projects Releases Packages Wiki Activity Actions

Hardware accelerated video decoding! Almost instant indexing! New single-file game format.

Browse source
This commit is contained in:
Scott Duensing 2026-09-04 13:31:38 -05:00
parent c0df1881a7
commit 84a2c25183
2094 changed files with 289552 additions and 697308 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

39
CHANGELOG
View file

@ -7,6 +7,35 @@ SINGE 3.00
API Changes
-----------
- A game can be one file: singe --pack DIRECTORY GAME.game writes the game's
scripts, art, sounds, fonts, and video into an SQLite database, and
singe GAME.game (or the menu, which lists every .game file beside the game
directories) runs it. Nothing in a game has to change to be packed:
every name a script uses (dofile, require, io.open and friends,
spriteLoad, soundLoad, fontLoad, videoLoad, framefiles, controls.cfg)
goes through one lookup that tries a loose directory named like the
database, then the game's data directory, then the database, so
packed games can be modded by dropping files beside them and files a
game writes land under data/<game>/files. Inside a database names
are matched without regard to case and a leading own-directory prefix
("DLe/Cfg/x") is ignored. singe --unpack GAME.game DIRECTORY restores
the files and singe --patch GAME.game SOURCE replaces files from a
directory or a patch database in one transaction. Loose games are
untouched by all of this. Scripts get SQLite for their own data with
require("sqlite3"). The archive installer (.game, .tool, and .patch
archives dropped beside the binary) is gone along with libarchive; a
game is a .game file or a directory, copied into place.
- Video is decoded by libavcodec directly; FFMS2 is gone. Seeking is still
frame exact, indexing is faster, and the index files in the data
directory use a new format (old ones are rebuilt automatically).
Multi-channel audio is downmixed to stereo instead of refused. Indexing
is a few seconds at most, so the indexing screen and its artwork are gone.
Video is
decoded by the platform's hardware decoder when there is one (VA-API or
VDPAU on Linux, D3D11VA on Windows, VideoToolbox on macOS) and falls
back to software otherwise; --softwarevideo forces software.
- Singe is built on SDL3 (SDL 3.4, SDL3_image, SDL3_mixer 3.2, SDL3_ttf)
instead of SDL2. Nothing changes for scripts: the same functions,
constants, scancodes, and controls.cfg names. SDL2_gfx is gone; sprite
@ -160,9 +189,19 @@ Fixes
video paths (the directory string was freed before use); a lone "." now
means the framefile's own directory.
- A crash prints a backtrace to the console (Linux and macOS) so a report
can say where it happened; run with --program and send trace.txt too.
- Mouse positions are delivered in the video's coordinates again (SDL3
stopped scaling them for us), so crosshairs follow the pointer.
- os.clock() in scripts now returns wall time rather than processor time.
Nearly every existing game times its input debounces and prompts with
it; once the GPU decodes the video the engine mostly sleeps, and those
timers ran seconds slow. A game that wanted processor time is not
known to exist. This diverges from stock Lua and from other Singe
runtimes; new code that must be portable should use singeGetTicks().
- A key or button still held when a game exits no longer counts as a
press in the script that follows. Confirming Exit in a game with
button 1 used to relaunch it from the menu.

46
CMakeLists.txt
View file

@ -125,9 +125,6 @@ singeEmbedImage(font)
singeEmbedImage(icon)
singeEmbedImage(kangarooPunchLogo)
singeEmbedImage(singeLogo)
singeEmbedImage(laserDisc)
singeEmbedImage(magnifyingGlass)
singeEmbedImage(indexing)
# Windows icon for the resource file.
add_custom_command(
@ -225,6 +222,8 @@ set(SINGE_SOURCE
src/frameFile.h
src/main.c
src/main.h
src/pack.c
src/pack.h
src/singe.c
src/singe.h
src/stddclmr.h
@ -232,6 +231,14 @@ set(SINGE_SOURCE
src/util.h
src/videoPlayer.c
src/videoPlayer.h
src/vfs.c
src/vfs.h
)
set(SQLITE_SOURCE
thirdparty/sqlite/sqlite3.c
thirdparty/sqlite/sqlite3.h
thirdparty/lsqlite3/lsqlite3.c
)
set(ARG_PARSER_SOURCE
@ -345,6 +352,7 @@ add_executable(${CMAKE_PROJECT_NAME}
${ARG_PARSER_SOURCE}
${LUA_SOURCE}
${LUA_FILESYSTEM_SOURCE}
${SQLITE_SOURCE}
${LUA_SOCKET_SOURCE}
${LUASEC_SOURCE}
${LUA_RS232_SOURCE}
@ -367,7 +375,10 @@ set_source_files_properties(${SINGE_SOURCE} PROPERTIES COMPILE_OPTIONS "-Wall;-W
target_compile_definitions(${CMAKE_PROJECT_NAME} PRIVATE
RS232_STATIC
FFMS_STATIC
SQLITE_DQS=0
SQLITE_THREADSAFE=1
SQLITE_DEFAULT_MEMSTATUS=0
SQLITE_OMIT_SHARED_CACHE
)
if(WIN32)
# _WIN32_WINNT=0x0600 sets the minimum compatible version of Windows to Vista.
@ -387,6 +398,7 @@ target_include_directories(${CMAKE_PROJECT_NAME} PRIVATE
${BUILD_DIR}
${BUILD_DIR}/include
thirdparty/lua/src
thirdparty/sqlite
thirdparty/luasec/src
thirdparty/librs232/include
)
@ -401,7 +413,7 @@ target_link_directories(${CMAKE_PROJECT_NAME} PRIVATE
# Platform specific system libraries.
if(KANGAROO_OS STREQUAL "linux")
# ffmpeg's vdpau hardware context is always compiled in on X11 hosts.
set(SYSTEM_LIBS -lX11 -lvdpau -ldl)
set(SYSTEM_LIBS -lX11 -lvdpau -ldl -rdynamic) # -rdynamic gives the crash backtrace function names
elseif(KANGAROO_OS STREQUAL "pi")
set(SYSTEM_LIBS -ldl)
elseif(KANGAROO_OS STREQUAL "macos")
@ -454,18 +466,33 @@ else()
message(FATAL_ERROR "Unknown KANGAROO_OS: ${KANGAROO_OS}")
endif()
# FFmpeg's hardware decoding backends (VA-API, VDPAU, DRM, V4L2) are whatever build-all.sh
# configured; its pkg-config files name the system libraries they need, so read them back.
if(KANGAROO_OS STREQUAL "linux" OR KANGAROO_OS STREQUAL "pi")
foreach(pc libavutil libavcodec)
set(pcFile ${BUILD_DIR}/lib/pkgconfig/${pc}.pc)
if(EXISTS ${pcFile})
file(STRINGS ${pcFile} pcLibs REGEX "^Libs:")
string(REGEX REPLACE "^Libs: *" "" pcLibs "${pcLibs}")
separate_arguments(pcLibs)
foreach(token ${pcLibs})
if(token MATCHES "^-l" AND NOT token MATCHES "^-l(avutil|avcodec|z|m|atomic|pthread)$")
list(APPEND SYSTEM_LIBS ${token})
endif()
endforeach()
endif()
endforeach()
list(REMOVE_DUPLICATES SYSTEM_LIBS)
endif()
# Static third party libraries built by build-all.sh.
set(STATIC_LIBS
${BUILD_DIR}/lib/libarchive.a
${BUILD_DIR}/lib/libavcodec.a
${BUILD_DIR}/lib/libavdevice.a
${BUILD_DIR}/lib/libavfilter.a
${BUILD_DIR}/lib/libavformat.a
${BUILD_DIR}/lib/libavutil.a
${BUILD_DIR}/lib/libbz2_static.a
${BUILD_DIR}/lib/libffms2.a
${BUILD_DIR}/lib/libfreetype.a
${BUILD_DIR}/lib/liblzma.a
${BUILD_DIR}/lib/libopus.a
${BUILD_DIR}/lib/libopusfile.a
${BUILD_DIR}/lib/libogg.a
@ -491,7 +518,6 @@ if(NOT KANGAROO_OS STREQUAL "macos")
list(APPEND STATIC_LIBS
${BUILD_DIR}/lib/libjpeg.a
${BUILD_DIR}/lib/libpng.a
-lstdc++
)
endif()

6
LICENSES
View file

@ -5,19 +5,17 @@ these tools, Singe would not exist.
arg_parser BSD-2-Clause http://savannah.nongnu.org/projects/arg-parser
binaryheap.lua MIT http://tieske.github.io/binaryheap.lua
bzip2 bzip2-1.0.6 https://sourceware.org/bzip2
copas MIT https://lunarmodules.github.io/copas
ffmpeg LGPL-2.1 https://ffmpeg.org
ffms2 GPL-3.0-only https://github.com/FFMS/ffms2
freetype FTL https://freetype.org
json.lua MIT https://github.com/rxi/json.lua
libarchive BSD-2-Clause https://libarchive.org
libjpeg-turbo IJG https://libjpeg-turbo.org
libogg BSD-3-Clause https://xiph.org/ogg
libpng libpng-2.0 http://www.libpng.org
librs232 MIT https://github.com/srdgame/librs232
libwebp BSD-3-Clause https://developers.google.com/speed/webp
libxmp MIT https://github.com/libxmp/libxmp
lsqlite3 MIT http://lua.sqlite.org
lua MIT https://www.lua.org
luafilesystem MIT https://lunarmodules.github.io/luafilesystem
luasec MIT https://github.com/lunarmodules/luasec
@ -30,11 +28,11 @@ SDL3 Zlib https://www.libsdl.org
SDL3_image Zlib https://www.libsdl.org
SDL3_mixer Zlib https://www.libsdl.org
SDL3_ttf Zlib https://www.libsdl.org
sqlite Public-Domain https://sqlite.org
timerwheel.lua MIT https://tieske.github.io/timerwheel.lua
uthash BSD-1-Clause https://troydhanson.github.io/uthash
vlc GPL-2.0 https://www.videolan.org/vlc
wavpack BSD-3-Clause https://www.wavpack.com
xz Public-Domain https://github.com/tukaani-project/xz
zlib Zlib https://zlib.net
zstd BSD-3-Clause https://facebook.github.io/zstd

56
assets/Menu.singe
View file

@ -409,7 +409,41 @@ function wrapText(text, maxWidth)
end
-- Search for games.dat files in subdirectories
-- Adds the entries of one games.dat. container is the game database the file came from, or nil.
local function loadGamesDat(source, container)
GAMES = {}
local ok, err = pcall(dofile, source)
if not ok then
debugPrint(source .. ": " .. tostring(err) .. " Skipped.")
GAMES = {}
return
end
for _, value in pairs(GAMES or {}) do
-- Since 3.00 a laserdisc game must say DISC = true; refuse the ambiguous cases here
-- with a message rather than letting the engine stop the menu when it is picked.
local title = tostring(value.TITLE or value.SCRIPT or "?")
if value.VIDEO and not value.DISC then
debugPrint(source .. ": \"" .. title .. "\" names a VIDEO without DISC = true; add DISC = true to the entry. Skipped.")
elseif value.DISC and not value.VIDEO then
debugPrint(source .. ": \"" .. title .. "\" says DISC = true but has no VIDEO. Skipped.")
else
if container then
-- Packed game: the engine opens its files through the database, and so does the menu.
value.CONTAINER = container
for _, key in ipairs({ "CABINET", "MARQUEE", "ATTRACT" }) do
if value[key] then
value[key] = container .. "/" .. value[key]
end
end
end
table.insert(GAME_LIST, value)
GAME_COUNT = GAME_COUNT + 1
end
end
GAMES = {}
end
-- Search for games.dat files in subdirectories and inside .game databases
GAME_LIST = {}
GAME_COUNT = 0
for dir in lfs.dir(".") do
@ -418,25 +452,11 @@ for dir in lfs.dir(".") do
if dirattr.mode == "directory" then
for file in lfs.dir(dir .. "/.") do
if file == "games.dat" then
-- Load games.dat
GAMES = {}
dofile(dir .. "/games.dat")
for _, value in pairs(GAMES or {}) do
-- Since 3.00 a laserdisc game must say DISC = true; refuse the ambiguous cases here
-- with a message rather than letting the engine stop the menu when it is picked.
local title = tostring(value.TITLE or value.SCRIPT or "?")
if value.VIDEO and not value.DISC then
debugPrint(dir .. "/games.dat: \"" .. title .. "\" names a VIDEO without DISC = true; add DISC = true to the entry. Skipped.")
elseif value.DISC and not value.VIDEO then
debugPrint(dir .. "/games.dat: \"" .. title .. "\" says DISC = true but has no VIDEO. Skipped.")
else
table.insert(GAME_LIST, value)
GAME_COUNT = GAME_COUNT + 1
end
end
GAMES = {}
loadGamesDat(dir .. "/games.dat", nil)
end
end
elseif dirattr.mode == "file" and dir:sub(-5):lower() == ".game" then
loadGamesDat(dir .. "/games.dat", dir)
end
end
end

96
build-all.sh
View file

@ -44,7 +44,14 @@ function buildAll() {
export CFLAGS="-I${G_TARGET}/include ${CFLAGS:-}"
export CXXFLAGS="-I${G_TARGET}/include ${CXXFLAGS:-}"
export LD_LIBRARY_PATH="${G_TARGET}/lib"
# Our own libraries first, then the platform's (FFmpeg's hardware decoders need libva and libdrm).
export PKG_CONFIG_LIBDIR="${G_TARGET}/lib/pkgconfig"
if [[ "${OS}" == "linux" ]]; then
export PKG_CONFIG_LIBDIR="${PKG_CONFIG_LIBDIR}:/usr/lib/${TRIPLE}/pkgconfig:/usr/lib/pkgconfig:/usr/share/pkgconfig"
elif [[ "${OS}" == "pi" ]]; then
export PKG_CONFIG_LIBDIR="${PKG_CONFIG_LIBDIR}:${SYSROOT}/usr/lib/${TRIPLE}/pkgconfig:${SYSROOT}/usr/lib/pkgconfig:${SYSROOT}/usr/share/pkgconfig"
export PKG_CONFIG_SYSROOT_DIR="${SYSROOT}"
fi
mkdir -p ${G_GENERATED}
@ -52,28 +59,9 @@ function buildAll() {
if [[ "${OS}" == "pi" ]]; then
export CFLAGS="--sysroot=${SYSROOT} ${CFLAGS}"
export CXXFLAGS="--sysroot=${SYSROOT} ${CXXFLAGS}"
sudo chroot ${SYSROOT} apt-get -y install libasound-dev libxi-dev libvdpau-dev
sudo chroot ${SYSROOT} apt-get -y install libasound-dev libxi-dev libvdpau-dev libdrm-dev
fi
pushd thirdparty/bzip2
clearAndEnterBuild
cmake ${COMMON} \
-DENABLE_SHARED_LIB=off \
-DENABLE_STATIC_LIB=on \
-DENABLE_LIB_ONLY=on \
..
make install
popd
pushd thirdparty/xz
clearAndEnterBuild
cmake ${COMMON} \
-DBUILD_SHARED_LIBS=off \
-DBUILD_TESTING=off \
..
make install
popd
pushd thirdparty/zlib
clearAndEnterBuild
CFLAGS="-I${PWD}" ../configure \
@ -166,42 +154,6 @@ function buildAll() {
make install
popd
pushd thirdparty/libarchive
clearAndEnterBuild
cmake ${COMMON} \
-DBUILD_SHARED_LIBS=off \
-DBUILD_TESTING=off \
-DENABLE_CAT=off \
-DENABLE_CNG=off \
-DENABLE_CPIO=off \
-DENABLE_EXPAT=off \
-DENABLE_ICONV=off \
-DENABLE_LIBB2=off \
-DENABLE_LIBGCC=off \
-DENABLE_LIBXML2=off \
-DENABLE_LZ4=off \
-DENABLE_LZO=off \
-DENABLE_MBEDTLS=off \
-DENABLE_NETTLE=off \
-DENABLE_OPENSSL=off \
-DENABLE_PCREPOSIX=off \
-DENABLE_TAR=off \
-DENABLE_TEST=off \
-DENABLE_UNZIP=off \
-DPOSIX_REGEX_LIB=libc \
-DUSE_BZIP2_STATIC=on \
-DBZIP2_INCLUDE_DIR="${G_TARGET}/include" \
-DBZIP2_LIBRARIES="${G_TARGET}/lib/libbz2_static.a" \
-DZLIB_ROOT="${G_TARGET}" \
-DLIBLZMA_INCLUDE_DIR="${G_TARGET}/include" \
-DLIBLZMA_LIBRARY="${G_TARGET}/lib/liblzma.a" \
-DZSTD_INCLUDE_DIR="${G_TARGET}/include" \
-DZSTD_LIBRARY="${G_TARGET}/lib/libzstd.a" \
-DWINDOWS_VERSION=VISTA \
..
make install
popd
pushd thirdparty/openssl
clearAndEnterBuild
if [[ "${OS}" == "windows" ]]; then
@ -252,6 +204,15 @@ function buildAll() {
mkdir -p "${G_TARGET}/include/sys"
echo "/* File no longer used */" > "${G_TARGET}/include/sys/sysctl.h"
fi
# Hardware decoding: each platform's native API. The libraries these pull in are read back
# from FFmpeg's pkg-config files by CMakeLists.txt, so the link follows this choice.
case "${OS}" in
linux) HWACCEL="--enable-vaapi --enable-vdpau --enable-libdrm" ;;
pi) HWACCEL="--enable-v4l2-m2m --enable-libdrm" ;;
macos) HWACCEL="--enable-videotoolbox" ;;
windows) HWACCEL="--enable-d3d11va --enable-dxva2" ;;
*) HWACCEL="" ;;
esac
pushd thirdparty/ffmpeg
clearAndEnterBuild
# https://trac.ffmpeg.org/wiki/CompilationGuide/CrossCompilingForWindows
@ -260,11 +221,13 @@ function buildAll() {
--disable-shared \
--disable-debug \
--disable-muxers \
--disable-hwaccels \
${HWACCEL} \
--disable-encoders \
--disable-filters \
--disable-network \
--disable-devices \
--disable-bzlib \
--disable-lzma \
--disable-doc \
--disable-programs \
--enable-gpl \
@ -280,21 +243,6 @@ function buildAll() {
make install
popd
pushd thirdparty/ffms2
# The configure script is not checked in; generate it once.
[[ -x configure ]] || NOCONFIGURE=1 ./autogen.sh
clearAndEnterBuild
../configure \
--prefix=${G_TARGET} \
--with-zlib=${G_TARGET} \
--enable-static \
--disable-shared \
--target=${TRIPLE} \
--host=${TRIPLE} \
--build=x86_64-linux
make install-libLTLIBRARIES # This weird target prevents building the command line tools.
popd
# Embedded resources, the version header, and the manual are generated by CMake.
pushd ${G_TARGET}
@ -341,7 +289,9 @@ sudo apt-get install -y \
libtool \
libasound-dev \
libxi-dev \
libvdpau-dev
libvdpau-dev \
libva-dev \
libdrm-dev
# Usage: build-all.sh [os arch] (default: every supported platform)

144
docs/Manual.adoc
View file

@ -69,9 +69,9 @@ be updated or deleted at any time.
=== Installing Games
Games designed for Singe 2.xx and later simply need to be unpacked and copied
into the same folder where you placed the Singe binary. The included menu
system will automatically detect them and add them to the menu.
A game is either a single `.game` file or a directory. Copy it into the same
folder where you placed the Singe binary. The included menu system will
automatically detect it and add it to the menu.
=== Customizing the Controls
@ -105,9 +105,10 @@ options available to be used in this file, read through
Singe [OPTIONS] scriptName{.singe}
----
The script name is the only required argument. It may be a `.singe` file, or
a directory containing a script of the same name (`ActionMax` finds
`ActionMax/ActionMax.singe`). For a laserdisc game (`--disc`) with no
The script name is the only required argument. It may be a `.singe` file, a
directory containing a script of the same name (`ActionMax` finds
`ActionMax/ActionMax.singe`), or a packed game (`DLe.game` runs the first
entry of the `games.dat` inside it). For a laserdisc game (`--disc`) with no
`--framefile`, Singe looks for a video next to the script with the same base
name and any extension FFmpeg can demux, then for a `.txt` framefile.
@ -122,15 +123,19 @@ name and any extension FFmpeg can demux, then for a `.txt` framefile.
| `-d`, `--datadir=PATHNAME` | Directory for everything Singe writes: video indexes, `trace.txt`, screenshots, the menu's `menu.dat`. A subdirectory named for the game's directory is created inside it. Defaults to the game's own directory.
| `-e`, `--volume_nonvldp=PERCENT` | Sound effect and extra video volume, `0` to `100`.
| `-f`, `--fullscreen` | Exclusive full screen at the desktop resolution.
| `-H`, `--softwarevideo` | Decode video in software even when the platform offers a hardware decoder (VA-API or VDPAU on Linux, D3D11VA on Windows, VideoToolbox on macOS). Use it to rule the hardware path in or out when a video misbehaves; the program trace says which decoder is in use.
| `-h`, `--help` | Show the option summary and exit.
| `-k`, `--nologos` | Skip the splash screens.
| `-l`, `--volume_vldp=PERCENT` | Laserdisc volume, `0` to `100`.
| `-m`, `--nomouse` | Disable mouse and lightgun input entirely.
| `-n`, `--nocrosshair` | Ask the game not to draw its own crosshair. Exposed to scripts as `singeWantsCrosshairs()`.
| `-o`, `--audio=TRACK` | Default audio track (zero based) for videos with several, such as multi-language releases.
| `-P`, `--pack=DIRECTORY` | Pack the game in `DIRECTORY` into the database named after the options, then exit. See <<singlefile,Single-File Games>>.
| `-p`, `--program` | Trace engine activity to the console and to `trace.txt` in the data directory.
| `-s`, `--nosound` | Mute all audio.
| `-T`, `--patch=GAME.game` | Replace files in the packed game from the directory or patch database named after the options, then exit.
| `-t`, `--trace` | Trace every Lua API call, with the script line that made it, to the console and to `trace.txt`.
| `-U`, `--unpack=GAME.game` | Write the packed game's files into the directory named after the options, then exit.
| `-u`, `--stretch` | Stretch the video to fill the window instead of keeping its aspect ratio.
| `-v`, `--framefile=FILENAME` | Video file or framefile to use instead of the one found next to the script.
| `-w`, `--fullscreen_window` | Borderless window covering the desktop.
@ -245,9 +250,9 @@ dofile("Singe/Framework.singe")
* Keep your game self-contained. If you build on a third-party framework or
share code between your games, copy it into your game directory. Never
reference a directory beside your game; the only file outside your game a
script may load is `Singe/Framework.singe`. A future single-file game format
packs exactly one directory, and a game that reaches outside it cannot be
packed.
script may load is `Singe/Framework.singe`. A single-file game (see
<<singlefile,Single-File Games>>) packs exactly one directory, and a game
that reaches outside it cannot be packed.
* Stay out of the `Singe/` folder. This is managed by Singe and anything added
or changed here is subject to future deletion.
* Include a `games.dat`. This is extremely important for new users. While
@ -276,6 +281,7 @@ ActionMax/ One game
38AmbushAlley.singe A script
frame_38AmbushAlley.txt Its framefile (or a video with the same base name)
sprite_*.png, sound_*.wav, font_*.ttf
DLe.game A game packed into one file (see Single-File Games)
data/
ActionMax/ Indexes, trace.txt, screenshots for that game
----
@ -294,32 +300,98 @@ so a game can live on read-only media.
=== Packaging Your Game
Singe installs games itself. Drop an archive into the installation directory,
start Singe, and the archive is validated, unpacked, and deleted. Any format
libarchive reads is accepted (zip, 7z, tar.gz, and so on); the extension
tells Singe what kind of package it is:
A finished game ships as one `.game` file, described next. Nothing else is
needed: the player copies the file beside the Singe binary and the menu
lists it. Singe no longer installs archives.
[cols="1,3",options="header"]
|===
| Extension | Rules
| `.game` | Everything must live inside one top level directory, which must contain a `games.dat`.
| `.tool` | Like a game, but no `games.dat` is required.
| `.patch` | Files may live anywhere; used to update an installed game in place.
|===
[#singlefile]
=== Single-File Games
Every package is rejected if it contains `controls.dat`, `Framework.singe`,
a file whose extension is `exe`, `sh`, `bat`, `cmd`, or `index`, or (for
games and tools) an extensionless file whose name starts with `singe`. Paths
that escape the installation directory are refused as well. Do not ship video
index files; Singe rebuilds them on first run.
To make a package, archive your game directory so the directory itself is the
top level entry, and rename the result:
A game can be one file. `singe --pack` writes a game directory into an
SQLite database, and Singe runs it from there: scripts, sprites, sounds,
fonts, framefiles, and video all load straight out of the database, and
installing the game is copying the file next to the executable. The menu
lists every `.game` file it finds beside the game directories.
----
zip -r ActionMax.game ActionMax
Singe --pack DLe DLe.game Pack the DLe directory into DLe.game
Singe DLe.game Run the first games.dat entry in it
Singe --unpack DLe.game DLe Write the files back out
Singe --patch DLe.game fixes Replace files from a directory (or a patch database)
----
Nothing in a game has to change to be packed. Every name a script uses,
whether through `dofile`, `require`, `io.open` and its relatives,
`spriteLoad`, `soundLoad`, `fontLoad`, `videoLoad`, a framefile, or a
`controls.cfg`, goes through one lookup. For a loose game that lookup is
the plain filesystem, relative to the directory Singe was started from,
exactly as before. For a packed game a name is tried in three places, in
this order, and the first hit wins:
. A loose directory named like the database without its extension
(`DLe/` beside `DLe.game`). This is how you keep editing a packed game,
and how a player mods one: files dropped there beat the packed copies.
. The game's data directory, under `files/`. Everything a game writes
with `io.output` or `io.open(name, "w")` lands here, so a saved
`game.cfg` beats the packed original and the database is never written.
. The database itself.
Inside a database, names are matched without regard to case, with either
path separator, and relative to the game root. A leading component equal to
the game's own directory name is ignored, so `DLe/Cfg/game.cfg`,
`Cfg/game.cfg`, and `DIR .. "Cfg/game.cfg"` all find the same file. Names
starting with `Singe/` or with the data directory stay on the filesystem;
`Singe/Framework.singe` and `singeGetDataPath()` work as always.
`require("name")` finds `name.lua`, `name/init.lua`, or `name.singe` under the
script's directory, then under the game root.
The packer refuses a directory that has no `games.dat`, that contains
`controls.dat`, `Framework.singe`, a file whose extension is `exe`, `sh`,
`bat`, or `cmd`, or an extensionless file whose name starts with `singe`,
that has a top level entry named like the directory itself (that would make the own-directory
rule ambiguous), or whose scripts and data files reach outside the game
with `..`; it reports the file and line. Two files whose names differ only
by case cannot both be packed. Stale `.index` files are skipped. A game
that references another game's directory (a shared framework beside it)
must copy that directory inside first.
`--patch` takes either a directory laid out like the game, whose files
replace the packed ones, or a database made by `--pack` from such a
directory. Either way the change is one transaction: the game is intact if
it is interrupted. A patch database may also carry a `removed(path)` table
naming files to delete.
A packed game keeps its data directory exactly where the loose game would
have it (`data/DLe/` for `DLe/DLe.singe`), so saves and settings are shared
between a loose install and a packed one. The video index goes there too.
When Singe runs a game from a database the menu passes `CONTAINER` in the
`games.dat` entry it launches; the field is set by the menu, never by hand.
For your own data, `require("sqlite3")` gives scripts the
https://lua.sqlite.org[lsqlite3] binding to the same SQLite the engine
uses. Keep such databases in `singeGetDataPath()`:
[source,lua]
----
local sqlite3 = require("sqlite3")
local db = sqlite3.open(singeGetDataPath() .. "scores.db")
db:exec("CREATE TABLE IF NOT EXISTS scores (name TEXT, score INTEGER)")
for row in db:nrows("SELECT name, score FROM scores ORDER BY score DESC LIMIT 10") do
print(row.name, row.score)
end
db:close()
----
The file format, for anyone writing tools: three tables.
`meta(key, value)` holds `version` (1), `gamedir` (the packed directory's
name), `chunk` (chunk size in bytes, 4194304), and `packer`.
`assets(path, name, size, data)` has one row per file, `path` being the
lower case forward-slash key and `name` the author's spelling; files no
larger than the chunk size are in `data`, larger ones have `data` NULL and
live in `chunks(path, chunk, data)` as consecutive numbered pieces. Singe
opens the database read only and immutable, so it runs from read-only
media and never journals beside it.
=== Event Driven... Or Not?
Traditionally, Singe used an event-driven programming model -- Singe handles
@ -697,6 +769,7 @@ install any additional software:
| LuaSec | TLS / SSL communication
| LuaSocket | TCP and UDP
| LuaRS232 | RS232 serial port access
| lsqlite3 | SQLite databases, `require("sqlite3")`
|===
Their usage is beyond the scope of this document.
@ -718,7 +791,11 @@ Two properties of the video file itself also matter. AAC audio carries encoder p
=== Video, Audio, and Container Formats
Singe decodes video with FFmpeg through FFMS2, so any container and codec the
Singe decodes video with FFmpeg's libraries directly, using the platform's hardware
decoder when it offers one for the codec (VA-API or VDPAU on Linux, D3D11VA on
Windows, VideoToolbox on macOS) and its own software decoder otherwise; the
program trace reports which, and `--softwarevideo` forces software. Any container
and codec the
bundled FFmpeg can demux and decode will play: MP4, MKV, MPEG program streams,
AVI, and the classic Daphne `.m2v` elementary streams with a matching `.ogg`
audio file next to them. Every audio track in the file is available to
@ -726,8 +803,9 @@ audio file next to them. Every audio track in the file is available to
format, channel count, and rate.
The first time a video is opened, Singe indexes it and stores the index next
to the game's other data (`<name>.index`). Indexing takes a while for large
files and happens again if the video changes. When a video is loaded, Singe
to the game's other data (`<name>.index`). Indexing is one pass over the file
without decoding, a few seconds even for a feature length disc, and happens
again if the video changes. When a video is loaded, Singe
reports its keyframe spacing in the program trace (`--program`), with a
warning there if keyframes are more than two seconds apart, because a seek
has to decode forward from the previous keyframe. Decoding happens on a separate thread,
@ -2177,7 +2255,7 @@ Returns the absolute path of the currently running script file. `Framework.singe
milliseconds = singeGetTicks()
----
Returns the wall clock in milliseconds since the engine started. Unlike `os.clock()`, which measures processor time and drifts whenever the engine idles, this is the clock to use for timers, debounces, and animation.
Returns the wall clock in milliseconds since the engine started. This is the clock to use for timers, debounces, and animation. Lua's `os.clock()` normally measures processor time, which drifts whenever the engine idles; because existing games use it as a wall clock, Singe replaces it with one that returns wall seconds since the engine started, so either call is safe.
*Since:* 3.00

3
src/embedded.h
View file

@ -29,9 +29,6 @@
#include "generated/icon.h"
#include "generated/kangarooPunchLogo.h"
#include "generated/singeLogo.h"
#include "generated/laserDisc.h"
#include "generated/magnifyingGlass.h"
#include "generated/indexing.h"
#include "generated/Framework_singe.h"
#include "generated/controls_cfg.h"
#include "generated/Menu_singe.h"

3
src/frameFile.c
View file

@ -27,6 +27,7 @@
#include "../thirdparty/uthash/src/uthash.h"
#include "util.h"
#include "vfs.h"
#include "videoPlayer.h"
#include "frameFile.h"
@ -162,7 +163,7 @@ int32_t frameFileLoad(const char *filename, const char *indexPath, SDL_Renderer
FrameLineT *newFiles = NULL;
FrameFileT *frameFile = NULL;
data = utilReadFile(filename, &bytes);
data = vfsRead(filename, &bytes);
if (!data) {
utilDie("Unable to open framefile: %s", filename);
}

2
src/frameFile.h
View file

@ -25,7 +25,7 @@
#define FRAMEFILE_H
#include <SDL2/SDL.h>
#include <SDL3/SDL.h>
#include "common.h"

411
src/main.c
View file

@ -26,7 +26,6 @@
#include <string.h>
#include <ctype.h>
#include <math.h>
#include <dirent.h>
#include <sys/stat.h>
#ifdef _WIN32
#include <io.h>
@ -34,8 +33,11 @@
#include <unistd.h>
#endif
#include "include/archive.h"
#include "include/archive_entry.h"
#include <signal.h>
#ifndef _WIN32
#include <execinfo.h>
#endif
#include <SDL3/SDL.h>
#include <SDL3/SDL_main.h>
#include <SDL3_image/SDL_image.h>
@ -51,6 +53,8 @@
#include "frameFile.h"
#include "videoPlayer.h"
#include "singe.h"
#include "pack.h"
#include "vfs.h"
#include "../thirdparty/ffmpeg/libavformat/avformat.h"
#include "embedded.h"
@ -61,17 +65,10 @@
#define MIXER_FREQUENCY 44100
#define MIXER_CHANNELS 2
#define MIXER_CHUNK_SAMPLES "1024" // Device buffer, kept small so the audio queue and any error in measuring it stay small
#define ARCHIVE_BLOCK_SIZE 10240
#define USAGE_OPTION_WIDTH 27
#define CRASH_FRAMES_MAX 64
typedef enum PackageTypeE {
PACKAGE_GAME = 0,
PACKAGE_TOOL,
PACKAGE_PATCH,
PACKAGE_COUNT
} PackageTypeE;
typedef struct RatioS {
int32_t aspectNum;
int32_t aspectDom;
@ -110,12 +107,6 @@ typedef struct EmbeddedFileS {
static QueueT *_scriptQueue = NULL;
static const char *const _packageTypes[PACKAGE_COUNT] = { "Game", "Tool", "Patch" };
static const char *const _badFilenames[] = { "controls.dat", "Framework.singe", NULL };
static const char *const _badExtensions[] = { "exe", "sh", "bat", "cmd", "index", NULL };
// Single source of truth for the command line: feeds both the parser and the usage text.
// The overscan and Sinden options work but are hidden until the Sinden border scales mouse input.
static const OptionT _options[] = {
@ -129,15 +120,19 @@ static const OptionT _options[] = {
{ 'e', "volume_nonvldp", ap_yes, "PERCENT", "specify sound effects volume in percent", false },
{ 'f', "fullscreen", ap_no, NULL, "run in full screen mode", false },
{ 'g', "sindengun", ap_yes, "'PARAMS'", "enable Sinden Light Gun support", true },
{ 'H', "softwarevideo", ap_no, NULL, "decode video in software even when a hardware decoder exists", false },
{ 'h', "help", ap_no, NULL, "this display", false },
{ 'k', "nologos", ap_no, NULL, "kill the splash screens", false },
{ 'l', "volume_vldp", ap_yes, "PERCENT", "specify laserdisc volume in percent", false },
{ 'm', "nomouse", ap_no, NULL, "disable mouse", false },
{ 'n', "nocrosshair", ap_no, NULL, "request game not display gun crosshairs", false },
{ 'o', "audio", ap_yes, "TRACK", "select default track for audio output", false },
{ 'P', "pack", ap_yes, "DIRECTORY", "pack a game DIRECTORY into the database named after the options", false },
{ 'p', "program", ap_no, NULL, "trace Singe execution to screen and file", false },
{ 's', "nosound", ap_no, NULL, "mutes all sound", false },
{ 'T', "patch", ap_yes, "DATABASE", "patch a game DATABASE from the directory or patch database named after the options", false },
{ 't', "trace", ap_no, NULL, "trace script execution to screen and file", false },
{ 'U', "unpack", ap_yes, "DATABASE", "unpack a game DATABASE into the directory named after the options", false },
{ 'u', "stretch", ap_no, NULL, "use ugly stretched video", false },
{ 'v', "framefile", ap_yes, "FILENAME", "use an alternate video file", false },
{ 'w', "fullscreen_window", ap_no, NULL, "run in windowed full screen mode", false },
@ -180,13 +175,15 @@ static const ModeT _modes[] = {
static char *_cloneString(const char *string);
static bool _extractArchive(const char *filename);
static bool _extractFile(const char *filename, const uint8_t *data, size_t length);
static char *_findVideoFile(const char *baseName);
#ifndef _WIN32
static void _crashHandler(int signalNumber);
#endif
static void _launcher(const char *exeName, ConfigT *conf);
static void _mainTrace(const ConfigT *conf, const char *fmt, ...) __attribute__((format(printf, 2, 3)));
static bool _runTool(const ConfigT *conf);
static bool _modeMatchesRatio(int32_t index, int32_t ratioIndex);
static struct archive *_openArchive(const char *filename);
static ConfigT *_parseArguments(const char *exeName, int32_t argc, char *argv[]);
static bool _parseInteger(const char *text, int32_t *value);
static void _resolveFiles(const char *exeName, ConfigT *conf);
@ -195,8 +192,6 @@ static void _showUsage(const char *name, const char *message) __attri
static void _startSDL(void);
static void _stopSDL(void);
static void _unpackData(const char *name);
static void _unpackGames(void);
static bool _validateArchive(const char *filename, PackageTypeE type);
static char *_cloneString(const char *string) {
@ -208,87 +203,6 @@ static char *_cloneString(const char *string) {
}
// https://github.com/libarchive/libarchive/wiki/Examples#user-content-A_Complete_Extractor
static bool _extractArchive(const char *filename) {
struct archive *a = NULL;
struct archive *ext = NULL;
struct archive_entry *entry = NULL;
const void *buff = NULL;
size_t size = 0;
la_int64_t offset = 0;
int32_t flags = 0;
int32_t r = 0;
bool ok = true;
// Never let an archive write outside the current directory.
flags = ARCHIVE_EXTRACT_TIME | ARCHIVE_EXTRACT_PERM | ARCHIVE_EXTRACT_ACL | ARCHIVE_EXTRACT_FFLAGS;
flags |= ARCHIVE_EXTRACT_SECURE_NODOTDOT | ARCHIVE_EXTRACT_SECURE_SYMLINKS | ARCHIVE_EXTRACT_SECURE_NOABSOLUTEPATHS;
a = _openArchive(filename);
if (a == NULL) {
return false;
}
ext = archive_write_disk_new();
archive_write_disk_set_options(ext, flags);
archive_write_disk_set_standard_lookup(ext);
while (ok) {
r = archive_read_next_header(a, &entry);
if (r == ARCHIVE_EOF) {
break;
}
if (r < ARCHIVE_OK) {
utilSay("%s", archive_error_string(a));
}
if (r < ARCHIVE_WARN) {
ok = false;
break;
}
r = archive_write_header(ext, entry);
if (r < ARCHIVE_OK) {
utilSay("%s", archive_error_string(ext));
} else {
if (archive_entry_size(entry) > 0) {
for (;;) {
r = archive_read_data_block(a, &buff, &size, &offset);
if (r == ARCHIVE_EOF) {
r = ARCHIVE_OK;
break;
}
if (r < ARCHIVE_OK) {
utilSay("%s", archive_error_string(a));
break;
}
r = archive_write_data_block(ext, buff, size, offset);
if (r < ARCHIVE_OK) {
utilSay("%s", archive_error_string(ext));
break;
}
}
if (r < ARCHIVE_WARN) {
ok = false;
break;
}
}
}
r = archive_write_finish_entry(ext);
if (r < ARCHIVE_OK) {
utilSay("%s", archive_error_string(ext));
}
if (r < ARCHIVE_WARN) {
ok = false;
break;
}
}
archive_read_close(a);
archive_read_free(a);
archive_write_close(ext);
archive_write_free(ext);
return ok;
}
// Writes an embedded support file, or rewrites it when the installed copy differs from this build's.
static bool _extractFile(const char *filename, const uint8_t *data, size_t length) {
FILE *out = NULL;
@ -346,7 +260,7 @@ static char *_findVideoFile(const char *baseName) {
comma++;
}
candidate = utilCreateString("%s.%s", baseName, extension);
if (utilFileExists(candidate)) {
if (vfsExists(candidate)) {
free(extensions);
return candidate;
}
@ -359,6 +273,22 @@ static char *_findVideoFile(const char *baseName) {
}
// Last words on a crash: where it happened, so a report can name the line. Async-signal-unsafe
// calls are acceptable here; the process is already lost.
#ifndef _WIN32
static void _crashHandler(int signalNumber) {
void *frames[CRASH_FRAMES_MAX];
int32_t count = backtrace(frames, CRASH_FRAMES_MAX);
fprintf(stderr, "\nSinge crashed (signal %d). Backtrace:\n", signalNumber);
backtrace_symbols_fd(frames, count, STDERR_FILENO);
fprintf(stderr, "Run with --program and send trace.txt with this.\n");
signal(signalNumber, SIG_DFL);
raise(signalNumber);
}
#endif
static void _launcher(const char *exeName, ConfigT *conf) {
int32_t x = 0;
int32_t bestResIndex = -1;
@ -495,6 +425,7 @@ static void _launcher(const char *exeName, ConfigT *conf) {
// Start our video playback system
_mainTrace(conf, "Initializing laserdisc video");
videoInit(mixer);
videoSetHardwareDecoding(!conf->softwareVideo);
// Finish our setup
_mainTrace(conf, "Disabling screen saver");
@ -536,21 +467,6 @@ static bool _modeMatchesRatio(int32_t index, int32_t ratioIndex) {
}
static struct archive *_openArchive(const char *filename) {
struct archive *a = archive_read_new();
archive_read_support_filter_all(a);
archive_read_support_format_all(a);
if (archive_read_open_filename(a, filename, ARCHIVE_BLOCK_SIZE) != ARCHIVE_OK) {
utilSay("!!! Cannot read %s: %s", filename, archive_error_string(a));
archive_read_free(a);
return NULL;
}
return a;
}
static ConfigT *_parseArguments(const char *exeName, int32_t argc, char *argv[]) {
int32_t x = 0;
int32_t argIndex = 0;
@ -668,6 +584,20 @@ static ConfigT *_parseArguments(const char *exeName, int32_t argc, char *argv[])
sindenString = strdup(arg);
break;
// Software video decoding
case 'H':
conf->softwareVideo = true;
break;
// Packing tools: the second name comes from the script argument
case 'P':
case 'T':
case 'U':
conf->toolMode = (code == 'P') ? TOOL_PACK : (code == 'T') ? TOOL_PATCH : TOOL_UNPACK;
free(conf->toolSource);
conf->toolSource = strdup(arg);
break;
// Help
case 'h':
_showUsage(exeName, NULL);
@ -760,9 +690,9 @@ static ConfigT *_parseArguments(const char *exeName, int32_t argc, char *argv[])
}
ap_free(&parser);
// A missing script is reported by main() after the support files and
// any game archives have been dealt with: running with no arguments
// is the documented way to install.
// A missing script is reported by main() after the support files have
// been dealt with: running with no arguments is the documented way to
// set up a fresh install.
// Do the full screen options make sense?
if (conf->fullScreen && conf->fullScreenWindow) {
@ -861,9 +791,10 @@ static void _resolveFiles(const char *exeName, ConfigT *conf) {
const char *extension = NULL;
char *temp = NULL;
// Exists?
// Exists? A packed game answers through its database.
vfsInit(conf->container, conf->dataDirBase, conf->dataDir);
utilFixPathSeparators(&conf->scriptFile, false);
if (!utilFileExists(conf->scriptFile)) {
if (!vfsExists(conf->scriptFile)) {
// Missing. Is a path?
temp = NULL;
if (utilPathExists(conf->scriptFile)) {
@ -884,7 +815,7 @@ static void _resolveFiles(const char *exeName, ConfigT *conf) {
// Do we need to generate a video name?
if (conf->videoFile) {
utilFixPathSeparators(&conf->videoFile, false);
if (!utilFileExists(conf->videoFile)) {
if (!vfsExists(conf->videoFile)) {
free(conf->videoFile);
conf->videoFile = NULL;
}
@ -900,7 +831,7 @@ static void _resolveFiles(const char *exeName, ConfigT *conf) {
// If we still don't have one, try a framefile
if (!conf->videoFile) {
conf->videoFile = utilCreateString("%s.txt", temp);
if (!utilFileExists(conf->videoFile)) {
if (!vfsExists(conf->videoFile)) {
free(conf->videoFile);
conf->videoFile = NULL;
}
@ -918,14 +849,11 @@ static void _resolveFiles(const char *exeName, ConfigT *conf) {
}
conf->isFrameFile = conf->disc && isFrameFileName(conf->videoFile);
if (conf->dataDir) {
// They provided a data directory. Append the game name.
conf->dataDirBase = conf->dataDir;
utilFixPathSeparators(&conf->dataDirBase, true);
conf->dataDir = createDataDir(conf->dataDirBase, conf->scriptFile);
if (conf->dataDirGiven || conf->container) {
// Under the base, in a directory named for the game.
conf->dataDir = createDataDirFor(conf);
} else {
// No data directory specified. Use the game folder.
conf->dataDirBase = utilCreateString(".%c", utilGetPathSeparator());
conf->dataDir = utilGetUpToLastPathComponent(conf->scriptFile);
}
if (!conf->dataDir) {
@ -951,6 +879,24 @@ static void _showHeader(void) {
}
// --pack, --unpack, and --patch: the option carries the source, the script argument the destination.
static bool _runTool(const ConfigT *conf) {
switch (conf->toolMode) {
case TOOL_PACK:
return packGame(conf->toolSource, conf->scriptFile);
case TOOL_PATCH:
return packPatch(conf->toolSource, conf->scriptFile);
case TOOL_UNPACK:
return packUnpack(conf->toolSource, conf->scriptFile);
default:
return false;
}
}
static void _showUsage(const char *name, const char *message) {
int32_t x = 0;
char *longForm = NULL;
@ -1057,148 +1003,6 @@ static void _unpackData(const char *name) {
}
// Games cannot include forbidden extensions, Singe binaries, multiple folders, etc.
// Tools are like games but do not need a games.dat file.
// Patches can be pretty much anything except forbidden extensions.
static void _unpackGames(void) {
struct dirent *de = NULL;
DIR *dir = opendir(".");
struct stat fileStat;
int32_t x = 0;
int32_t count = 0;
PackageTypeE type = PACKAGE_COUNT;
if (dir == NULL) {
utilDie("Could not open the current directory.");
}
while ((de = readdir(dir)) != NULL) {
// Only regular files with a package extension are interesting.
if ((strcmp(de->d_name, ".") == 0) || (strcmp(de->d_name, "..") == 0)) {
continue;
}
if ((stat(de->d_name, &fileStat) != 0) || !S_ISREG(fileStat.st_mode)) {
continue;
}
type = PACKAGE_COUNT;
for (x = 0; x < PACKAGE_COUNT; x++) {
if (utilStricmp(utilGetFileExtension(de->d_name), _packageTypes[x]) == 0) {
type = (PackageTypeE)x;
break;
}
}
if (type == PACKAGE_COUNT) {
continue;
}
_showHeader();
count++;
if (_validateArchive(de->d_name, type)) {
utilSay(">>> Installing %s: %s", _packageTypes[type], de->d_name);
if (_extractArchive(de->d_name)) {
unlink(de->d_name);
}
}
}
if (count > 0) {
utilNewline();
}
closedir(dir);
}
// Look through archive for things I've told people NOT to ship!
// https://github.com/libarchive/libarchive/wiki/Examples#user-content-List_contents_of_Archive_stored_in_File
static bool _validateArchive(const char *filename, PackageTypeE type) {
struct archive *a = NULL;
struct archive_entry *entry = NULL;
const char *path = NULL;
const char *name = NULL;
const char *extension = NULL;
const char *slash = NULL;
const char *typeName = _packageTypes[type];
char *toplevel = NULL;
char *gamesDat = NULL;
int32_t x = 0;
bool ok = true;
bool hasGamesDat = false;
a = _openArchive(filename);
if (a == NULL) {
return false;
}
while (ok && (archive_read_next_header(a, &entry) == ARCHIVE_OK)) {
path = archive_entry_pathname(entry);
name = utilGetLastPathComponent(path);
extension = utilGetFileExtension(path);
// Everything except a patch must live inside a single top level folder.
if (type != PACKAGE_PATCH) {
if (toplevel == NULL) {
slash = strchr(path, '/');
if (slash == NULL) {
ok = false;
utilSay("!!! %s has files in root: %s", typeName, filename);
} else {
toplevel = utilStrndup(path, (size_t)(slash - path + 1));
gamesDat = utilCreateString("%sgames.dat", toplevel);
}
} else {
if (!utilStartsWith(path, toplevel)) {
ok = false;
if (strchr(path, '/') == NULL) {
utilSay("!!! %s has files in root: %s", typeName, filename);
} else {
utilSay("!!! %s has multiple top level directories: %s", typeName, filename);
}
}
}
}
if (ok) {
// Is this a forbidden file?
for (x = 0; ok && (_badFilenames[x] != NULL); x++) {
if (utilStricmp(name, _badFilenames[x]) == 0) {
ok = false;
utilSay("!!! %s has %s: %s", typeName, _badFilenames[x], filename);
}
}
// Is this a forbidden extension?
for (x = 0; ok && (_badExtensions[x] != NULL); x++) {
if (utilStricmp(extension, _badExtensions[x]) == 0) {
ok = false;
utilSay("!!! %s has %s file: %s", typeName, _badExtensions[x], filename);
}
}
// Not a patch, no extension, starts with 'singe' - could be unix binary.
if (ok && (type != PACKAGE_PATCH) && (strlen(extension) == 0) && utilStartsWith(name, "singe")) {
ok = false;
utilSay("!!! %s has singe file: %s", typeName, filename);
}
// Is this games.dat?
if (ok && (gamesDat != NULL) && (utilStricmp(path, gamesDat) == 0)) {
hasGamesDat = true;
}
}
archive_read_data_skip(a);
}
archive_read_free(a);
free(toplevel);
free(gamesDat);
// If it's a game, did we get a games.dat?
if (ok && (type == PACKAGE_GAME) && !hasGamesDat) {
ok = false;
utilSay("!!! %s has no games.dat: %s", typeName, filename);
}
return ok;
}
ConfigT *cloneConf(const ConfigT *conf) {
ConfigT *c = (ConfigT *)calloc(1, sizeof(ConfigT));
@ -1209,6 +1013,8 @@ ConfigT *cloneConf(const ConfigT *conf) {
// Copy everything, then give the clone its own strings.
*c = *conf;
c->scriptFile = _cloneString(conf->scriptFile);
c->container = _cloneString(conf->container);
c->toolSource = _cloneString(conf->toolSource);
c->videoFile = _cloneString(conf->videoFile);
c->dataDirBase = _cloneString(conf->dataDirBase);
c->dataDir = _cloneString(conf->dataDir);
@ -1253,6 +1059,24 @@ char *createDataDir(const char *dataDirBase, const char *filename) {
}
// The data directory follows the script's directory, or the database's name when the script sits at its root.
char *createDataDirFor(const ConfigT *conf) {
const char *base = NULL;
char *name = NULL;
char *path = NULL;
if ((conf->container != NULL) && (strchr(conf->scriptFile, '/') == NULL) && (strchr(conf->scriptFile, '\\') == NULL)) {
base = utilGetLastPathComponent(conf->container);
name = utilCreateString("%.*s%c%s", (int)(strlen(base) - strlen(VFS_DATABASE_EXTENSION)), base, utilGetPathSeparator(), conf->scriptFile);
path = createDataDir(conf->dataDirBase, name);
free(name);
return path;
}
return createDataDir(conf->dataDirBase, conf->scriptFile);
}
void destroyConf(ConfigT **confPointer) {
ConfigT *conf = *confPointer;
@ -1263,6 +1087,8 @@ void destroyConf(ConfigT **confPointer) {
free(conf->dataDirBase);
free(conf->videoFile);
free(conf->scriptFile);
free(conf->container);
free(conf->toolSource);
free(conf);
*confPointer = NULL;
}
@ -1332,7 +1158,16 @@ int main(int argc, char *argv[]) {
const char *exeName = argv[0];
char *temp = NULL;
ConfigT *conf = NULL;
ConfigT *replacement = NULL;
QueueT *q = NULL;
bool ok = false;
#ifndef _WIN32
signal(SIGSEGV, _crashHandler);
signal(SIGBUS, _crashHandler);
signal(SIGABRT, _crashHandler);
signal(SIGFPE, _crashHandler);
#endif
// Options first so --help and --noconsole take effect before anything is written.
conf = _parseArguments(exeName, argc, argv);
@ -1341,13 +1176,40 @@ int main(int argc, char *argv[]) {
utilRedirectConsole();
_unpackData(exeName);
_unpackGames();
// -d names the base under which every game gets a data directory; without it the game folder serves.
if (conf->dataDir) {
conf->dataDirBase = conf->dataDir;
conf->dataDir = NULL;
conf->dataDirGiven = true;
utilFixPathSeparators(&conf->dataDirBase, true);
} else {
conf->dataDirBase = utilCreateString(".%c", utilGetPathSeparator());
}
// The packing tools need no window: run one and leave.
if (conf->toolMode != TOOL_NONE) {
if (!conf->scriptFile) {
_showUsage(exeName, "The packing tools need a second name after the options.");
}
ok = _runTool(conf);
destroyConf(&conf);
vfsQuit();
return ok ? EXIT_SUCCESS : EXIT_FAILURE;
}
// Nothing to run? Installing was the whole job.
if (!conf->scriptFile) {
_showUsage(exeName, "No script file specified.");
}
// A game database on its own runs its first games.dat entry.
if (vfsIsDatabase(conf->scriptFile)) {
replacement = confFromDatabase(conf);
destroyConf(&conf);
conf = replacement;
}
// Queue initial script
_resolveFiles(exeName, conf);
queueScript(conf);
@ -1376,6 +1238,7 @@ int main(int argc, char *argv[]) {
}
_stopSDL();
vfsQuit();
if (utilGetConsoleEnabled()) {
utilWaitForKeyOnWindows();

1
src/main.h
View file

@ -39,6 +39,7 @@
ConfigT *cloneConf(const ConfigT *conf);
char *createDataDir(const char *dataDirBase, const char *filename);
char *createDataDirFor(const ConfigT *conf);
void destroyConf(ConfigT **confPointer);
bool isFrameFileName(const char *filename);
bool parseSindenString(const char *sindenString, ConfigT *conf);

609
src/pack.c Normal file
View file

@ -0,0 +1,609 @@
/*
* Packer, unpacker, and patcher for single-file games. The database layout is described in vfs.c.
*/
#include <ctype.h>
#include <dirent.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <strings.h>
#include <sys/stat.h>
#include "sqlite3.h"
#include "pack.h"
#include "singe.h"
#include "util.h"
#include "vfs.h"
#define PAGE_SIZE 4096
typedef struct EntryS {
char *relative; // Forward slashes, author's case
char *full; // Filesystem path
int64_t size;
struct EntryS *next;
} EntryT;
typedef struct WriterS {
sqlite3 *db;
sqlite3_stmt *assetStmt;
sqlite3_stmt *chunkStmt;
sqlite3_stmt *deleteChunkStmt;
int64_t chunkBytes;
int64_t files;
int64_t bytes;
} WriterT;
static const char *const _badFilenames[] = { "controls.dat", "Framework.singe", NULL };
static const char *const _badExtensions[] = { "exe", "sh", "bat", "cmd", NULL };
static const char *const _textExtensions[] = { "singe", "lua", "dat", "cfg", "txt", NULL };
static bool _applyDatabase(WriterT *writer, const char *source);
static bool _applyDirectory(WriterT *writer, const char *directory, bool checkGame);
static bool _collect(const char *root, const char *relative, EntryT **list);
static bool _createTables(sqlite3 *db);
static void _entriesFree(EntryT *list);
static bool _exec(sqlite3 *db, const char *sql);
static bool _insertFile(WriterT *writer, const EntryT *entry);
static bool _isDirectory(const char *path);
static char *_keyFor(const char *relative);
static bool _scanEscapes(const EntryT *list);
static bool _writeMeta(sqlite3 *db, const char *key, const char *value);
static bool _writerClose(WriterT *writer, bool commit);
static bool _writerOpen(WriterT *writer, const char *database, bool create);
// Copies every asset row of a patch database into the open game, replacing what it names.
static bool _applyDatabase(WriterT *writer, const char *source) {
sqlite3_stmt *stmt = NULL;
char *sql = utilCreateString("ATTACH DATABASE '%s' AS patch", source);
int64_t chunk = 0;
bool ok = true;
if (!_exec(writer->db, sql)) {
free(sql);
return false;
}
free(sql);
if (sqlite3_prepare_v2(writer->db, "SELECT value FROM patch.meta WHERE key = 'chunk'", -1, &stmt, NULL) == SQLITE_OK) {
if (sqlite3_step(stmt) == SQLITE_ROW) {
chunk = sqlite3_column_int64(stmt, 0);
}
sqlite3_finalize(stmt);
}
if (chunk != writer->chunkBytes) {
utilSay("!!! %s uses a different chunk size than the game.", source);
return false;
}
ok = ok && _exec(writer->db, "DELETE FROM main.chunks WHERE path IN (SELECT path FROM patch.assets)");
ok = ok && _exec(writer->db, "INSERT OR REPLACE INTO main.assets (path, name, size, data) SELECT path, name, size, data FROM patch.assets");
ok = ok && _exec(writer->db, "INSERT OR REPLACE INTO main.chunks (path, chunk, data) SELECT path, chunk, data FROM patch.chunks");
if (ok && (sqlite3_prepare_v2(writer->db, "SELECT count(*) FROM patch.sqlite_master WHERE name = 'removed'", -1, &stmt, NULL) == SQLITE_OK)) {
if ((sqlite3_step(stmt) == SQLITE_ROW) && (sqlite3_column_int64(stmt, 0) > 0)) {
ok = ok && _exec(writer->db, "DELETE FROM main.chunks WHERE path IN (SELECT path FROM patch.removed)");
ok = ok && _exec(writer->db, "DELETE FROM main.assets WHERE path IN (SELECT path FROM patch.removed)");
}
sqlite3_finalize(stmt);
}
if (ok && (sqlite3_prepare_v2(writer->db, "SELECT count(*), coalesce(sum(size), 0) FROM patch.assets", -1, &stmt, NULL) == SQLITE_OK)) {
if (sqlite3_step(stmt) == SQLITE_ROW) {
writer->files = sqlite3_column_int64(stmt, 0);
writer->bytes = sqlite3_column_int64(stmt, 1);
}
sqlite3_finalize(stmt);
}
return ok;
}
// Inserts every file below directory. checkGame enforces the rules a whole game must meet.
static bool _applyDirectory(WriterT *writer, const char *directory, bool checkGame) {
EntryT *list = NULL;
EntryT *entry = NULL;
char *gameDir = NULL;
char *reason = NULL;
const char *slash = NULL;
size_t length = 0;
bool ok = true;
if (!_collect(directory, "", &list)) {
return false;
}
gameDir = strdup(utilGetLastPathComponent(directory));
for (entry = list; ok && (entry != NULL); entry = entry->next) {
reason = packForbiddenReason(entry->relative, !checkGame);
if (reason != NULL) {
utilSay("!!! %s has %s: %s", directory, reason, entry->relative);
free(reason);
ok = false;
}
// A top level entry named like the game directory would make the own-directory prefix ambiguous.
if (ok && checkGame) {
slash = strchr(entry->relative, '/');
length = slash ? (size_t)(slash - entry->relative) : strlen(entry->relative);
if ((length == strlen(gameDir)) && (strncasecmp(entry->relative, gameDir, length) == 0)) {
utilSay("!!! %s contains an entry named like the game directory (%s); rename one of them.", directory, gameDir);
ok = false;
}
}
}
if (ok && checkGame && !_scanEscapes(list)) {
ok = false;
}
for (entry = list; ok && (entry != NULL); entry = entry->next) {
ok = _insertFile(writer, entry);
}
free(gameDir);
_entriesFree(list);
return ok;
}
// Recursive directory walk producing relative names with forward slashes. Stale index files are skipped.
static bool _collect(const char *root, const char *relative, EntryT **list) {
DIR *dir = NULL;
struct dirent *de = NULL;
struct stat info;
char *full = NULL;
char *child = NULL;
EntryT *entry = NULL;
bool ok = true;
full = utilCreateString("%s%s%s", root, relative[0] ? "/" : "", relative);
dir = opendir(full);
if (dir == NULL) {
utilSay("!!! Unable to read directory %s", full);
free(full);
return false;
}
while (ok && ((de = readdir(dir)) != NULL)) {
if ((strcmp(de->d_name, ".") == 0) || (strcmp(de->d_name, "..") == 0)) {
continue;
}
child = utilCreateString("%s%s%s", relative, relative[0] ? "/" : "", de->d_name);
free(full);
full = utilCreateString("%s/%s", root, child);
if (stat(full, &info) != 0) {
utilSay("!!! Unable to stat %s", full);
ok = false;
} else if (S_ISDIR(info.st_mode)) {
ok = _collect(root, child, list);
} else if (S_ISREG(info.st_mode)) {
if (utilStricmp(utilGetFileExtension(de->d_name), "index") == 0) {
utilSay(">>> Skipping stale index %s", child);
} else {
entry = (EntryT *)calloc(1, sizeof(EntryT));
entry->relative = strdup(child);
entry->full = strdup(full);
entry->size = (int64_t)info.st_size;
entry->next = *list;
*list = entry;
}
}
free(child);
}
closedir(dir);
free(full);
return ok;
}
static bool _createTables(sqlite3 *db) {
bool ok = true;
ok = ok && _exec(db, "CREATE TABLE meta (key TEXT PRIMARY KEY, value TEXT)");
ok = ok && _exec(db, "CREATE TABLE assets (path TEXT PRIMARY KEY, name TEXT, size INTEGER, data BLOB)");
ok = ok && _exec(db, "CREATE TABLE chunks (path TEXT, chunk INTEGER, data BLOB, PRIMARY KEY (path, chunk))");
return ok;
}
static void _entriesFree(EntryT *list) {
EntryT *next = NULL;
while (list != NULL) {
next = list->next;
free(list->relative);
free(list->full);
free(list);
list = next;
}
}
static bool _exec(sqlite3 *db, const char *sql) {
char *error = NULL;
if (sqlite3_exec(db, sql, NULL, NULL, &error) != SQLITE_OK) {
utilSay("!!! SQLite: %s", error);
sqlite3_free(error);
return false;
}
return true;
}
// One file becomes one assets row, plus chunk rows when it is larger than the chunk size.
static bool _insertFile(WriterT *writer, const EntryT *entry) {
FILE *file = NULL;
uint8_t *buffer = NULL;
char *key = _keyFor(entry->relative);
size_t got = 0;
int64_t index = 0;
bool ok = true;
file = fopen(entry->full, "rb");
if (file == NULL) {
utilSay("!!! Unable to read %s", entry->full);
free(key);
return false;
}
buffer = (uint8_t *)malloc((size_t)writer->chunkBytes);
sqlite3_reset(writer->deleteChunkStmt);
sqlite3_bind_text(writer->deleteChunkStmt, 1, key, -1, SQLITE_STATIC);
sqlite3_step(writer->deleteChunkStmt);
sqlite3_reset(writer->assetStmt);
sqlite3_bind_text(writer->assetStmt, 1, key, -1, SQLITE_STATIC);
sqlite3_bind_text(writer->assetStmt, 2, entry->relative, -1, SQLITE_STATIC);
sqlite3_bind_int64(writer->assetStmt, 3, entry->size);
if (entry->size <= writer->chunkBytes) {
got = fread(buffer, 1, (size_t)entry->size, file);
sqlite3_bind_blob(writer->assetStmt, 4, buffer, (int)got, SQLITE_STATIC);
} else {
sqlite3_bind_null(writer->assetStmt, 4);
}
ok = (sqlite3_step(writer->assetStmt) == SQLITE_DONE);
sqlite3_reset(writer->assetStmt);
if (ok && (entry->size > writer->chunkBytes)) {
while (ok && ((got = fread(buffer, 1, (size_t)writer->chunkBytes, file)) > 0)) {
sqlite3_reset(writer->chunkStmt);
sqlite3_bind_text(writer->chunkStmt, 1, key, -1, SQLITE_STATIC);
sqlite3_bind_int64(writer->chunkStmt, 2, index);
sqlite3_bind_blob(writer->chunkStmt, 3, buffer, (int)got, SQLITE_STATIC);
ok = (sqlite3_step(writer->chunkStmt) == SQLITE_DONE);
sqlite3_reset(writer->chunkStmt);
index++;
}
}
if (!ok) {
utilSay("!!! SQLite: %s", sqlite3_errmsg(writer->db));
}
writer->files++;
writer->bytes += entry->size;
fclose(file);
free(buffer);
free(key);
return ok;
}
static bool _isDirectory(const char *path) {
struct stat info;
return (stat(path, &info) == 0) && S_ISDIR(info.st_mode);
}
// The lookup key: lower case, forward slashes.
static char *_keyFor(const char *relative) {
char *key = strdup(relative);
char *p = NULL;
for (p = key; *p != 0; p++) {
*p = (*p == '\\') ? '/' : (char)tolower((unsigned char)*p);
}
return key;
}
// A packed game may only reach its own contents: ".." in a script or data file is refused, by line.
static bool _scanEscapes(const EntryT *list) {
const EntryT *entry = NULL;
const char *ext = NULL;
const char *offset = NULL;
char *data = NULL;
char *line = NULL;
size_t bytes = 0;
int32_t number = 0;
int32_t x = 0;
bool text = false;
bool ok = true;
for (entry = list; entry != NULL; entry = entry->next) {
ext = utilGetFileExtension(entry->relative);
text = false;
for (x = 0; !text && (_textExtensions[x] != NULL); x++) {
text = (utilStricmp(ext, _textExtensions[x]) == 0);
}
if (!text) {
continue;
}
data = utilReadFile(entry->full, &bytes);
if (data == NULL) {
continue;
}
offset = data;
number = 0;
while ((line = utilReadLine(data, bytes, &offset)) != NULL) {
number++;
if ((strstr(line, "../") != NULL) || (strstr(line, "..\\") != NULL)) {
utilSay("!!! %s:%d reaches outside the game with \"..\"", entry->relative, number);
ok = false;
}
free(line);
}
free(data);
}
return ok;
}
static bool _writeMeta(sqlite3 *db, const char *key, const char *value) {
sqlite3_stmt *stmt = NULL;
bool ok = false;
if (sqlite3_prepare_v2(db, "INSERT OR REPLACE INTO meta (key, value) VALUES (?, ?)", -1, &stmt, NULL) == SQLITE_OK) {
sqlite3_bind_text(stmt, 1, key, -1, SQLITE_STATIC);
sqlite3_bind_text(stmt, 2, value, -1, SQLITE_STATIC);
ok = (sqlite3_step(stmt) == SQLITE_DONE);
}
sqlite3_finalize(stmt);
return ok;
}
static bool _writerClose(WriterT *writer, bool commit) {
bool ok = true;
if (writer->db == NULL) {
return false;
}
ok = _exec(writer->db, commit ? "COMMIT" : "ROLLBACK");
sqlite3_finalize(writer->assetStmt);
sqlite3_finalize(writer->chunkStmt);
sqlite3_finalize(writer->deleteChunkStmt);
sqlite3_close(writer->db);
memset(writer, 0, sizeof(*writer));
return ok;
}
// Opens the database for writing inside one transaction; create replaces any existing file.
static bool _writerOpen(WriterT *writer, const char *database, bool create) {
sqlite3_stmt *stmt = NULL;
char *text = NULL;
bool ok = true;
memset(writer, 0, sizeof(*writer));
writer->chunkBytes = VFS_CHUNK_BYTES;
if (create) {
if (utilFileExists(database)) {
utilSay(">>> Replacing %s", database);
remove(database);
}
} else if (!utilFileExists(database)) {
utilSay("!!! %s does not exist.", database);
return false;
}
if (sqlite3_open_v2(database, &writer->db, SQLITE_OPEN_READWRITE | (create ? SQLITE_OPEN_CREATE : 0), NULL) != SQLITE_OK) {
utilSay("!!! Unable to open %s: %s", database, sqlite3_errmsg(writer->db));
sqlite3_close(writer->db);
writer->db = NULL;
return false;
}
text = utilCreateString("PRAGMA page_size = %d", PAGE_SIZE);
ok = ok && _exec(writer->db, text);
free(text);
ok = ok && _exec(writer->db, "PRAGMA journal_mode = OFF");
ok = ok && _exec(writer->db, "PRAGMA synchronous = OFF");
if (ok && create) {
ok = _createTables(writer->db);
}
if (ok && !create) {
// An existing game must be one of ours, and its chunk size rules.
if ((sqlite3_prepare_v2(writer->db, "SELECT value FROM meta WHERE key = 'chunk'", -1, &stmt, NULL) != SQLITE_OK) || (sqlite3_step(stmt) != SQLITE_ROW)) {
utilSay("!!! %s is not a Singe game database.", database);
ok = false;
} else {
writer->chunkBytes = sqlite3_column_int64(stmt, 0);
}
sqlite3_finalize(stmt);
}
ok = ok && _exec(writer->db, "BEGIN");
ok = ok && (sqlite3_prepare_v2(writer->db, "INSERT OR REPLACE INTO assets (path, name, size, data) VALUES (?, ?, ?, ?)", -1, &writer->assetStmt, NULL) == SQLITE_OK);
ok = ok && (sqlite3_prepare_v2(writer->db, "INSERT OR REPLACE INTO chunks (path, chunk, data) VALUES (?, ?, ?)", -1, &writer->chunkStmt, NULL) == SQLITE_OK);
ok = ok && (sqlite3_prepare_v2(writer->db, "DELETE FROM chunks WHERE path = ?", -1, &writer->deleteChunkStmt, NULL) == SQLITE_OK);
if (!ok) {
utilSay("!!! Unable to prepare %s: %s", database, sqlite3_errmsg(writer->db));
sqlite3_finalize(writer->assetStmt);
sqlite3_finalize(writer->chunkStmt);
sqlite3_finalize(writer->deleteChunkStmt);
sqlite3_close(writer->db);
memset(writer, 0, sizeof(*writer));
}
return ok;
}
// Why a file may not ship in a game, as a new string, or NULL when it may. Shared with the archive installer.
char *packForbiddenReason(const char *path, bool isPatch) {
const char *name = utilGetLastPathComponent(path);
const char *extension = utilGetFileExtension(path);
int32_t x = 0;
for (x = 0; _badFilenames[x] != NULL; x++) {
if (utilStricmp(name, _badFilenames[x]) == 0) {
return strdup(_badFilenames[x]);
}
}
for (x = 0; _badExtensions[x] != NULL; x++) {
if (utilStricmp(extension, _badExtensions[x]) == 0) {
return utilCreateString("%s file", _badExtensions[x]);
}
}
// No extension and starts with "singe": could be a unix binary.
if (!isPatch && (strlen(extension) == 0) && utilStartsWith(name, "singe")) {
return strdup("singe file");
}
return NULL;
}
// Packs a game directory into a new database.
bool packGame(const char *directory, const char *database) {
WriterT writer;
char *root = strdup(directory);
char *gamesDat = NULL;
char *chunk = NULL;
size_t length = strlen(root);
bool ok = true;
while ((length > 1) && ((root[length - 1] == '/') || (root[length - 1] == '\\'))) {
root[--length] = 0;
}
if (!_isDirectory(root)) {
utilSay("!!! %s is not a directory.", root);
free(root);
return false;
}
gamesDat = utilCreateString("%s/games.dat", root);
if (!utilFileExists(gamesDat)) {
utilSay("!!! %s has no games.dat; a game database needs one at its root.", root);
free(gamesDat);
free(root);
return false;
}
free(gamesDat);
if (!_writerOpen(&writer, database, true)) {
free(root);
return false;
}
chunk = utilCreateString("%lld", (long long)writer.chunkBytes);
ok = ok && _writeMeta(writer.db, "version", "1");
ok = ok && _writeMeta(writer.db, "gamedir", utilGetLastPathComponent(root));
ok = ok && _writeMeta(writer.db, "chunk", chunk);
ok = ok && _writeMeta(writer.db, "packer", VERSION_STRING);
free(chunk);
ok = ok && _applyDirectory(&writer, root, true);
if (ok) {
utilSay(">>> Packed %lld files (%lld bytes) from %s into %s", (long long)writer.files, (long long)writer.bytes, root, database);
}
if (!_writerClose(&writer, ok)) {
ok = false;
}
if (!ok) {
remove(database);
}
free(root);
return ok;
}
// Applies a patch, either a directory of replacement files or a patch database, in one transaction.
bool packPatch(const char *database, const char *source) {
WriterT writer;
bool ok = true;
if (!_writerOpen(&writer, database, false)) {
return false;
}
if (_isDirectory(source)) {
ok = _applyDirectory(&writer, source, false);
} else if (utilFileExists(source)) {
ok = _applyDatabase(&writer, source);
} else {
utilSay("!!! %s does not exist.", source);
ok = false;
}
if (ok) {
utilSay(">>> Patched %s with %lld files (%lld bytes) from %s", database, (long long)writer.files, (long long)writer.bytes, source);
}
if (!_writerClose(&writer, ok)) {
ok = false;
}
return ok;
}
// Writes every asset back out as files, in the author's spelling, under directory.
bool packUnpack(const char *database, const char *directory) {
sqlite3 *db = NULL;
sqlite3_stmt *assets = NULL;
sqlite3_stmt *chunks = NULL;
FILE *file = NULL;
char *path = NULL;
char *parent = NULL;
const char *name = NULL;
int64_t files = 0;
bool ok = true;
if (!vfsIsDatabase(database)) {
utilSay("!!! %s is not a Singe game database.", database);
return false;
}
if (sqlite3_open_v2(database, &db, SQLITE_OPEN_READONLY, NULL) != SQLITE_OK) {
utilSay("!!! Unable to open %s: %s", database, sqlite3_errmsg(db));
sqlite3_close(db);
return false;
}
ok = ok && (sqlite3_prepare_v2(db, "SELECT path, name, data FROM assets ORDER BY path", -1, &assets, NULL) == SQLITE_OK);
ok = ok && (sqlite3_prepare_v2(db, "SELECT data FROM chunks WHERE path = ? ORDER BY chunk", -1, &chunks, NULL) == SQLITE_OK);
while (ok && (sqlite3_step(assets) == SQLITE_ROW)) {
name = (const char *)sqlite3_column_text(assets, 1);
path = utilCreateString("%s/%s", directory, name ? name : (const char *)sqlite3_column_text(assets, 0));
utilFixPathSeparators(&path, false);
parent = utilGetUpToLastPathComponent(path);
if (!utilMkDirP(parent, 0755)) {
utilSay("!!! Unable to create %s", parent);
ok = false;
} else {
file = fopen(path, "wb");
if (file == NULL) {
utilSay("!!! Unable to write %s", path);
ok = false;
} else {
if (sqlite3_column_type(assets, 2) != SQLITE_NULL) {
fwrite(sqlite3_column_blob(assets, 2), 1, (size_t)sqlite3_column_bytes(assets, 2), file);
} else {
sqlite3_reset(chunks);
sqlite3_bind_text(chunks, 1, (const char *)sqlite3_column_text(assets, 0), -1, SQLITE_TRANSIENT);
while (sqlite3_step(chunks) == SQLITE_ROW) {
fwrite(sqlite3_column_blob(chunks, 0), 1, (size_t)sqlite3_column_bytes(chunks, 0), file);
}
}
fclose(file);
files++;
}
}
free(parent);
free(path);
}
sqlite3_finalize(assets);
sqlite3_finalize(chunks);
sqlite3_close(db);
if (ok) {
utilSay(">>> Unpacked %lld files from %s into %s", (long long)files, database, directory);
}
return ok;
}

16
src/pack.h Normal file
View file

@ -0,0 +1,16 @@
/*
* Packing games into, and out of, single-file SQLite databases.
*/
#ifndef PACK_H
#define PACK_H
#include <stdbool.h>
char *packForbiddenReason(const char *path, bool isPatch);
bool packGame(const char *directory, const char *database);
bool packPatch(const char *database, const char *source);
bool packUnpack(const char *database, const char *directory);
#endif

413
src/singe.c
View file

@ -47,12 +47,15 @@ LUASOCKET_API int luaopen_socket_serial(lua_State *L);
#endif
// There is no header for rs232 binding. Make our own.
int luaopen_luars232(lua_State *L);
// Nor for lsqlite3.
int luaopen_lsqlite3(lua_State *L);
// There is no header for ssl.config binding. Make our own.
LSEC_API int luaopen_ssl_config(lua_State *L);
#include "main.h"
#include "util.h"
#include "frameFile.h"
#include "vfs.h"
#include "videoPlayer.h"
#include "singe.h"
@ -64,9 +67,6 @@ LSEC_API int luaopen_ssl_config(lua_State *L);
#include "embedded.h"
#define INDEX_DISPLAY_START -1
#define INDEX_DISPLAY_STOP -2
// soundSetVolume/soundGetVolume use this legacy scale; the mixer uses MIX_MAX_VOLUME.
#define AUDIO_MAX_VOLUME 63
#define MAX_MICE 4
@ -100,6 +100,8 @@ LSEC_API int luaopen_ssl_config(lua_State *L);
#define ANIMATION_MIN_DELAY_MS 10 // GIFs often carry a zero delay
#define SCREENSHOT_MAX 10000
#define SOUND_QUEUE_SIZE 64
#define MS_PER_SECOND_NUMBER 1000.0
#define INPUT_GRACE_MS 1000 // Presses this soon after a script starts or focus arrives were held over from before
#define EFFECT_TRACKS 16 // Sound effect "channels" scripts can play at once
#define EFFECT_TAG "effects"
#define HELD_KEYS_MAX 64 // Keys physically down at once
@ -112,13 +114,6 @@ LSEC_API int luaopen_ssl_config(lua_State *L);
#define LOGO_FADE_STEP_MS 3
#define LOGO_HOLD_MS 750
#define INDEX_SCREEN_WIDTH 1280
#define INDEX_SCREEN_HEIGHT 720
#define INDEX_RADIUS_FACTOR 0.3
#define INDEX_UPDATE_MS 5
#define INDEX_TEXT_OVERLAP 5
#define INDEX_GLASS_X_FACTOR 0.15
#define INDEX_GLASS_Y_FACTOR 0.1
// Decoded video frames are BGRA in memory; this is the matching alpha-less surface format.
#if SDL_BYTEORDER == SDL_LIL_ENDIAN
@ -293,6 +288,7 @@ typedef struct GlobalS {
bool keyboardState[SDL_SCANCODE_COUNT];
bool keySuppressed[SDL_SCANCODE_COUNT]; // Held before this script started; not a press for it
bool buttonSuppressed[MAX_CONTROLLERS][CONTROLLER_BUTTON_COUNT];
uint64_t inputGraceUntil; // Until then, new presses are treated as held over too
int32_t keyboardLastDown;
int32_t keyboardLastUp;
int32_t frameFileHandle;
@ -349,6 +345,8 @@ static MIX_Track *_effectTracks[EFFECT_TRACKS];
static const LuaModuleT _luaModules[] = {
// LuaFileSystem
MODC("lfs", luaopen_lfs),
// SQLite for script data
MODC("sqlite3", luaopen_lsqlite3),
// LuaSocket
MODC("mime.core", luaopen_mime_core),
MODC("socket.core", luaopen_socket_core),
@ -436,7 +434,7 @@ static SoundT *_argSound(lua_State *L, const char *method, int32_t index);
static SpriteT *_argSprite(lua_State *L, const char *method, int32_t index);
static const char *_argString(lua_State *L, const char *method, int32_t index);
static VideoT *_argVideo(lua_State *L, const char *method, int32_t index);
static ConfigT *_buildConfFromTable(lua_State *L);
static ConfigT *_buildConfFromTable(lua_State *L, const ConfigT *base);
static void _callLua(const char *func, const char *sig, ...);
static int32_t _effectTrackFree(void);
static void _effectStopped(void *userdata, MIX_Track *track);
@ -444,7 +442,6 @@ static int32_t _controllerSlot(SDL_JoystickID which);
static bool _delayAndPump(uint32_t ms);
static void _deliverKey(bool down, int32_t keysym, int32_t scancode);
static void _discSeek(int64_t frame);
static void _doIndexDisplay(int32_t percent);
static void _doLogos(void);
static void _drawLine(int32_t x1, int32_t y1, int32_t x2, int32_t y2, uint32_t pixel);
static void _drawPauseIndicator(const SDL_FRect *target);
@ -453,12 +450,18 @@ static void _fireMouseMoved(int32_t device, int32_t x, int32_t y, int32_
static void _fontDestroy(FontT *font);
static void _freezeGame(bool freeze);
static void _heldListUpdate(HeldKeyT *list, int32_t *count, bool down, int32_t keysym, int32_t scancode);
static void _installFileHooks(lua_State *L);
static int32_t _loadAudioCalibration(void);
static void _loadControlsFile(const char *path);
static SDL_Surface *_loadEmbeddedPng(const unsigned char *data, unsigned int length);
static SDL_Texture *_loadEmbeddedTexture(const unsigned char *data, unsigned int length, SDL_Surface **surface);
static void _luaDie(lua_State *L, const char *method, const char *fmt, ...) __attribute__((format(printf, 3, 4))) __attribute__((noreturn));
static int32_t _luaDofile(lua_State *L);
static int32_t _luaFileSearcher(lua_State *L);
static char *_luaFormat(lua_State *L, const char *method, const char *fmt, va_list args);
static int32_t _luaIoHook(lua_State *L);
static int32_t _luaLoadfile(lua_State *L);
static int32_t _luaLoadFile(lua_State *L, const char *name, const char *mode);
static int32_t _luaPanic(lua_State *L);
static int32_t _luaSearcher(lua_State *L);
static void _luaTrace(lua_State *L, const char *method, const char *fmt, ...) __attribute__((format(printf, 3, 4)));
@ -538,6 +541,7 @@ static int32_t apiMouseHowMany(lua_State *L);
static int32_t apiMouseSetCaptured(lua_State *L);
static int32_t apiMouseSetEnabled(lua_State *L);
static int32_t apiMouseSetMode(lua_State *L);
static int32_t apiOsClock(lua_State *L);
static int32_t apiOverlayBox(lua_State *L);
static int32_t apiOverlayCircle(lua_State *L);
static int32_t apiOverlayClear(lua_State *L);
@ -733,15 +737,17 @@ static VideoT *_argVideo(lua_State *L, const char *method, int32_t index) {
// Builds a config for scriptExecute/scriptPush from the games.dat style table at stack index 1.
static ConfigT *_buildConfFromTable(lua_State *L) {
static ConfigT *_buildConfFromTable(lua_State *L, const ConfigT *base) {
const char *confKey = NULL;
const char *valueString = NULL;
bool valueBoolean = false;
int64_t valueNumber = 0;
ConfigT *c = NULL;
// Start with current config, but every entry declares its own disc.
c = cloneConf(_global.conf);
// Start with the given config, but every entry declares its own disc and container.
c = cloneConf(base);
free(c->container);
c->container = NULL;
c->disc = false;
c->isFrameFile = false;
free(c->videoFile);
@ -786,6 +792,13 @@ static ConfigT *_buildConfFromTable(lua_State *L) {
free(c->scriptFile);
c->scriptFile = strdup(valueString);
utilFixPathSeparators(&c->scriptFile, false);
} else if (strcmp(confKey, "CONTAINER") == 0) {
if (valueString == NULL) {
utilDie("CONTAINER must be a string.");
}
free(c->container);
c->container = strdup(valueString);
utilFixPathSeparators(&c->container, false);
} else if (strcmp(confKey, "VIDEO") == 0) {
if (valueString == NULL) {
utilDie("VIDEO must be a string.");
@ -837,7 +850,7 @@ static ConfigT *_buildConfFromTable(lua_State *L) {
// Create new data dir location based on script location.
free(c->dataDir);
c->dataDir = createDataDir(c->dataDirBase, c->scriptFile);
c->dataDir = createDataDirFor(c);
if (!c->dataDir) {
utilDie("Unable to create data directory for %s.", c->scriptFile);
}
@ -1029,95 +1042,6 @@ static void _discSeek(int64_t frame) {
}
// Indexing progress display. INDEX_DISPLAY_START sets up, 0..100 animates, INDEX_DISPLAY_STOP tears down.
static void _doIndexDisplay(int32_t percent) {
static int32_t oldW = 0;
static int32_t oldH = 0;
static int32_t radius = 0;
static int32_t angle = 0;
static uint64_t nextUpdate = 0;
static SDL_RendererLogicalPresentation oldMode = SDL_LOGICAL_PRESENTATION_DISABLED;
static SDL_Surface *surfDisc = NULL;
static SDL_Surface *surfGlass = NULL;
static SDL_Surface *surfIndex = NULL;
static SDL_Texture *texDisc = NULL;
static SDL_Texture *texGlass = NULL;
static SDL_Texture *texIndex = NULL;
SDL_FRect target;
int32_t vShift = 0;
(void)percent;
if (percent == INDEX_DISPLAY_START) {
SDL_GetRenderLogicalPresentation(_global.renderer, &oldW, &oldH, &oldMode);
SDL_SetRenderLogicalPresentation(_global.renderer, INDEX_SCREEN_WIDTH, INDEX_SCREEN_HEIGHT, SDL_LOGICAL_PRESENTATION_LETTERBOX);
texGlass = _loadEmbeddedTexture(magnifyingGlass_png, magnifyingGlass_png_len, &surfGlass);
texDisc = _loadEmbeddedTexture(laserDisc_png, laserDisc_png_len, &surfDisc);
texIndex = _loadEmbeddedTexture(indexing_png, indexing_png_len, &surfIndex);
radius = (int32_t)(surfDisc->w * INDEX_RADIUS_FACTOR);
nextUpdate = SDL_GetTicks() + INDEX_UPDATE_MS;
return;
}
if (percent == INDEX_DISPLAY_STOP) {
SDL_DestroyTexture(texDisc);
SDL_DestroyTexture(texGlass);
SDL_DestroyTexture(texIndex);
SDL_DestroySurface(surfDisc);
SDL_DestroySurface(surfGlass);
SDL_DestroySurface(surfIndex);
texDisc = NULL;
texGlass = NULL;
texIndex = NULL;
surfDisc = NULL;
surfGlass = NULL;
surfIndex = NULL;
SDL_SetRenderLogicalPresentation(_global.renderer, oldW, oldH, oldMode);
SDL_SetRenderDrawColor(_global.renderer, 0, 0, 0, SDL_ALPHA_OPAQUE);
SDL_RenderClear(_global.renderer);
SDL_RenderPresent(_global.renderer);
return;
}
// Display animation. Indexing runs on the main thread, so keep the window's events flowing
// or the renderer never learns about a size change (fullscreen is applied asynchronously).
SDL_PumpEvents();
SDL_SetRenderLogicalPresentation(_global.renderer, INDEX_SCREEN_WIDTH, INDEX_SCREEN_HEIGHT, SDL_LOGICAL_PRESENTATION_LETTERBOX);
SDL_SetRenderDrawColor(_global.renderer, 0, 0, 0, SDL_ALPHA_OPAQUE);
SDL_RenderClear(_global.renderer);
// Draw "Indexing" text
vShift = surfIndex->h - INDEX_TEXT_OVERLAP;
target.x = (float)((INDEX_SCREEN_WIDTH - surfIndex->w) / 2);
target.y = (float)(INDEX_SCREEN_HEIGHT - vShift);
target.w = (float)surfIndex->w;
target.h = (float)surfIndex->h;
SDL_RenderTexture(_global.renderer, texIndex, NULL, &target);
// Draw laserdisc
target.x = (float)((INDEX_SCREEN_WIDTH - surfDisc->w) / 2);
target.y = (float)((INDEX_SCREEN_HEIGHT - surfDisc->h) / 2 - vShift);
target.w = (float)surfDisc->w;
target.h = (float)surfDisc->h;
SDL_RenderTexture(_global.renderer, texDisc, NULL, &target);
// Draw magnifying glass circling the disc
target.x = (float)(int32_t)(surfDisc->w * INDEX_GLASS_X_FACTOR + (INDEX_SCREEN_WIDTH - surfGlass->w) / 2 + cos(angle * M_PI / (DEGREES_PER_CIRCLE / 2)) * radius);
target.y = (float)(int32_t)(surfDisc->h * INDEX_GLASS_Y_FACTOR + (INDEX_SCREEN_HEIGHT - surfGlass->h) / 2 - vShift + sin(angle * M_PI / (DEGREES_PER_CIRCLE / 2)) * radius);
target.w = (float)surfGlass->w;
target.h = (float)surfGlass->h;
SDL_RenderTexture(_global.renderer, texGlass, NULL, &target);
// Update animation
if (SDL_GetTicks() > nextUpdate) {
angle = (angle + 1) % (int32_t)DEGREES_PER_CIRCLE;
nextUpdate = SDL_GetTicks() + INDEX_UPDATE_MS;
}
SDL_RenderPresent(_global.renderer);
}
// Splash screens: fade in the Kangaroo Punch logo, cross fade to the Singe logo, fade out.
static void _doLogos(void) {
int32_t i = 0;
@ -1356,6 +1280,32 @@ static void _heldListUpdate(HeldKeyT *list, int32_t *count, bool down, int32_t k
}
// Replaces dofile, loadfile, and the io functions that take a file name with versions that resolve
// the name through the vfs. Each closure keeps the original as its first upvalue.
static void _installFileHooks(lua_State *L) {
static const struct {
const char *name;
int32_t mode; // 0 read, 1 io.open (mode string decides), 2 write
} ioHooks[] = { { "input", 0 }, { "lines", 0 }, { "open", 1 }, { "output", 2 } };
size_t i = 0;
lua_getglobal(L, "dofile");
lua_pushcclosure(L, _luaDofile, 1);
lua_setglobal(L, "dofile");
lua_getglobal(L, "loadfile");
lua_pushcclosure(L, _luaLoadfile, 1);
lua_setglobal(L, "loadfile");
lua_getglobal(L, "io");
for (i = 0; i < sizeof(ioHooks) / sizeof(ioHooks[0]); i++) {
lua_getfield(L, -1, ioHooks[i].name);
lua_pushinteger(L, ioHooks[i].mode);
lua_pushcclosure(L, _luaIoHook, 2);
lua_setfield(L, -2, ioHooks[i].name);
}
lua_pop(L, 1);
}
// The per-machine audio delay lives beside the data directories, since it is not a property of any game.
static int32_t _loadAudioCalibration(void) {
char *path = utilCreateString("%s%s", _global.conf->dataDirBase, AUDIO_CALIBRATION_FILE);
@ -1380,9 +1330,9 @@ static int32_t _loadAudioCalibration(void) {
// Runs a controls.cfg if it exists.
static void _loadControlsFile(const char *path) {
if (utilFileExists(path)) {
if (vfsExists(path)) {
_progTrace("Loading %s", path);
if (luaL_dofile(_global.luaContext, path)) {
if (_luaLoadFile(_global.luaContext, path, NULL) || lua_pcall(_global.luaContext, 0, 0, 0)) {
utilDie("%s", lua_tostring(_global.luaContext, -1));
}
}
@ -1431,6 +1381,73 @@ static void _luaDie(lua_State *L, const char *method, const char *fmt, ...) {
// Formats "line:method: message" for tracing and errors. Caller frees.
// dofile(name) through the vfs; without a name the original reads stdin.
static int32_t _luaDofile(lua_State *L) {
const char *name = luaL_optstring(L, 1, NULL);
if (name == NULL) {
lua_pushvalue(L, lua_upvalueindex(1));
lua_insert(L, 1);
lua_call(L, lua_gettop(L) - 1, LUA_MULTRET);
return lua_gettop(L);
}
lua_settop(L, 1);
if (_luaLoadFile(L, name, NULL) != LUA_OK) {
return lua_error(L);
}
lua_call(L, 0, LUA_MULTRET);
return lua_gettop(L) - 1;
}
// require() of a game's own module: name.lua, name/init.lua, or name.singe under the script's
// directory, then relative to the game root.
static int32_t _luaFileSearcher(lua_State *L) {
static const char *const patterns[] = { "%s%s.lua", "%s%s/init.lua", "%s%s.singe", NULL };
char *module = strdup(lua_tostring(L, 1));
char *scriptDir = utilGetUpToLastPathComponent(_global.conf->scriptFile);
const char *prefixes[2];
char *name = NULL;
char *p = NULL;
int32_t x = 0;
int32_t y = 0;
prefixes[0] = scriptDir;
prefixes[1] = "";
for (p = module; *p != 0; p++) {
if (*p == '.') {
*p = '/';
}
}
for (y = 0; y < 2; y++) {
for (x = 0; patterns[x] != NULL; x++) {
name = utilCreateString(patterns[x], prefixes[y], module);
if (vfsExists(name)) {
if (_luaLoadFile(L, name, NULL) != LUA_OK) {
lua_pushfstring(L, "error loading module '%s' from file '%s':\n\t%s", lua_tostring(L, 1), name, lua_tostring(L, -1));
free(name);
free(module);
free(scriptDir);
return lua_error(L);
}
lua_pushstring(L, name);
free(name);
free(module);
free(scriptDir);
return 2;
}
free(name);
}
}
lua_pushfstring(L, "\n\tno file '%s%s.lua' in the game", scriptDir, module);
free(module);
free(scriptDir);
return 1;
}
static char *_luaFormat(lua_State *L, const char *method, const char *fmt, va_list args) {
lua_Debug ar;
int32_t line = 0;
@ -1455,6 +1472,78 @@ static char *_luaFormat(lua_State *L, const char *method, const char *fmt, va_li
// Lua panic handler: something went wrong outside a protected call.
// io.open and friends with a name resolved through the vfs. Upvalues: the original, and how the name is used.
static int32_t _luaIoHook(lua_State *L) {
int32_t mode = (int32_t)lua_tointeger(L, lua_upvalueindex(2));
const char *modeString = NULL;
char *path = NULL;
bool writing = (mode == 2);
int32_t top = lua_gettop(L);
if (lua_type(L, 1) == LUA_TSTRING) {
if (mode == 1) {
modeString = luaL_optstring(L, 2, "r");
writing = (strpbrk(modeString, "wa+") != NULL);
}
path = vfsFilePath(lua_tostring(L, 1), writing);
lua_pushstring(L, path);
lua_replace(L, 1);
free(path);
}
lua_pushvalue(L, lua_upvalueindex(1));
lua_insert(L, 1);
lua_call(L, top, LUA_MULTRET);
return lua_gettop(L);
}
// loadfile(name, mode, env) through the vfs; without a name the original reads stdin.
static int32_t _luaLoadfile(lua_State *L) {
const char *name = luaL_optstring(L, 1, NULL);
const char *mode = luaL_optstring(L, 2, NULL);
if (name == NULL) {
lua_pushvalue(L, lua_upvalueindex(1));
lua_insert(L, 1);
lua_call(L, lua_gettop(L) - 1, LUA_MULTRET);
return lua_gettop(L);
}
if (_luaLoadFile(L, name, mode) != LUA_OK) {
lua_pushnil(L);
lua_insert(L, -2);
return 2;
}
if (!lua_isnone(L, 3)) {
lua_pushvalue(L, 3);
if (lua_setupvalue(L, -2, 1) == NULL) {
lua_pop(L, 1);
}
}
return 1;
}
// Loads a chunk from the vfs, leaving the function or an error message on the stack. Returns the Lua status.
static int32_t _luaLoadFile(lua_State *L, const char *name, const char *mode) {
char *data = NULL;
char *chunkName = NULL;
size_t bytes = 0;
int32_t status = LUA_ERRFILE;
data = vfsRead(name, &bytes);
if (data == NULL) {
lua_pushfstring(L, "cannot open %s", name);
return status;
}
chunkName = utilCreateString("@%s", name);
status = luaL_loadbufferx(L, data, bytes, chunkName, mode);
free(chunkName);
free(data);
return status;
}
static int32_t _luaPanic(lua_State *L) {
lua_Debug ar;
int32_t level = 0;
@ -1475,6 +1564,8 @@ static int32_t _luaPanic(lua_State *L) {
// package.searchers entry serving the embedded Lua modules.
// https://leiradel.github.io/2020/03/01/Embedding-Lua-Modules.html
static int32_t _luaSearcher(lua_State *L) {
const char *modname = lua_tostring(L, 1);
size_t i = 0;
@ -1937,18 +2028,30 @@ static void _startLuaContext(lua_State *L) {
// Register the standard libraries
luaL_openlibs(L);
// Games use os.clock() as a wall clock for debounces and timers, and it is processor time: with
// the GPU decoding video the engine mostly sleeps, so those timers crawl. Give them what they meant.
lua_getglobal(L, "os");
lua_pushcfunction(L, apiOsClock);
lua_setfield(L, -2, "clock");
lua_pop(L, 1);
// Every file a script names goes through the vfs.
_installFileHooks(L);
_pushConstants(L);
// Put our searcher at the front of package.searchers.
// Put our searchers at the front of package.searchers: embedded modules, then game files.
lua_getglobal(L, "package");
lua_getfield(L, -1, "searchers");
length = lua_rawlen(L, -1);
for (i = length + 1; i > 1; i--) {
lua_rawgeti(L, -2, (lua_Integer)(i - 1));
for (i = length + 2; i > 2; i--) {
lua_rawgeti(L, -2, (lua_Integer)(i - 2));
lua_rawseti(L, -2, (lua_Integer)i);
}
lua_pushcfunction(L, _luaSearcher);
lua_rawseti(L, -2, 1);
lua_pushcfunction(L, _luaFileSearcher);
lua_rawseti(L, -2, 2);
lua_pop(L, 2);
}
@ -1979,6 +2082,10 @@ static void _suppressHeldInput(void) {
int32_t b = 0;
const bool *state = SDL_GetKeyboardState(&count);
// SDL may not know about a held key or button yet (gamepads are polled on their own thread, and
// X11 re-reports keys after the focus event), so presses that arrive soon after count as held too.
_global.inputGraceUntil = SDL_GetTicks() + INPUT_GRACE_MS;
for (x = 0; x < SDL_SCANCODE_COUNT; x++) {
_global.keySuppressed[x] = (x < count) && state[x];
}
@ -2534,7 +2641,7 @@ static int32_t apiFontLoad(lua_State *L) {
if (!font) {
_luaDie(L, "fontLoad", "Unable to allocate new font.");
}
font->font = TTF_OpenFont(name, (float)points);
font->font = TTF_OpenFontIO(vfsOpenIO(name), true, (float)points);
if (!font->font) {
_luaDie(L, "fontLoad", "%s", SDL_GetError());
}
@ -2771,6 +2878,14 @@ static int32_t apiMouseSetMode(lua_State *L) {
}
// seconds = os.clock() Replaces Lua's processor-time clock with wall time since the engine started.
static int32_t apiOsClock(lua_State *L) {
lua_pushnumber(L, (lua_Number)SDL_GetTicks() / MS_PER_SECOND_NUMBER);
return 1;
}
// overlayBox(x1, y1, x2, y2) Outline only.
static int32_t apiOverlayBox(lua_State *L) {
int32_t x1 = 0;
@ -3077,7 +3192,7 @@ static int32_t apiScriptExecute(lua_State *L) {
if (!lua_istable(L, 1)) {
_luaDie(L, "scriptExecute", "Argument 1 must be a table.");
}
conf = _buildConfFromTable(L);
conf = _buildConfFromTable(L, _global.conf);
queueScript(conf);
destroyConf(&conf);
_global.running = false;
@ -3095,7 +3210,7 @@ static int32_t apiScriptPush(lua_State *L) {
if (!lua_istable(L, 1)) {
_luaDie(L, "scriptPush", "Argument 1 must be a table.");
}
conf = _buildConfFromTable(L);
conf = _buildConfFromTable(L, _global.conf);
queueScript(conf);
destroyConf(&conf);
queueScript(_global.conf);
@ -3353,7 +3468,7 @@ static int32_t apiSoundLoad(lua_State *L) {
if (!sound) {
_luaDie(L, "soundLoad", "Unable to allocate new sound.");
}
sound->audio = MIX_LoadAudio(videoGetMixer(), name, true);
sound->audio = MIX_LoadAudio_IO(videoGetMixer(), vfsOpenIO(name), true, true);
if (!sound->audio) {
_luaDie(L, "soundLoad", "%s", SDL_GetError());
}
@ -3610,6 +3725,7 @@ static int32_t apiSpriteIsPlaying(lua_State *L) {
static int32_t apiSpriteLoad(lua_State *L) {
const char *name = NULL;
SpriteT *sprite = NULL;
SDL_IOStream *io = NULL;
int32_t x = 0;
_argCheck(L, "spriteLoad", 1, 1);
@ -3619,7 +3735,11 @@ static int32_t apiSpriteLoad(lua_State *L) {
_luaDie(L, "spriteLoad", "Unable to allocate new sprite.");
}
// Try to load requested file as an animation first
sprite->animation = IMG_LoadAnimation(name);
io = vfsOpenIO(name);
if (io == NULL) {
_luaDie(L, "spriteLoad", "%s", SDL_GetError());
}
sprite->animation = IMG_LoadAnimation_IO(io, false);
if ((sprite->animation != NULL) && (sprite->animation->count < 2)) {
// Only one frame - keep it as a still image.
sprite->originalSurface = _surfaceCopy(sprite->animation->frames[0]);
@ -3634,10 +3754,12 @@ static int32_t apiSpriteLoad(lua_State *L) {
}
sprite->originalSurface = sprite->animation->frames[0];
} else {
sprite->originalSurface = IMG_Load(name);
SDL_SeekIO(io, 0, SDL_IO_SEEK_SET);
sprite->originalSurface = IMG_Load_IO(io, false);
_surfaceUnpack(&sprite->originalSurface);
}
}
SDL_CloseIO(io);
if (!sprite->originalSurface) {
_luaDie(L, "spriteLoad", "%s", SDL_GetError());
}
@ -4303,6 +4425,44 @@ static int32_t apiVldpSetVerbose(lua_State *L) {
// ===== Engine entry point =====
// A game database named on the command line runs the first entry of its games.dat.
ConfigT *confFromDatabase(const ConfigT *conf) {
lua_State *L = luaL_newstate();
ConfigT *base = cloneConf(conf);
ConfigT *result = NULL;
base->container = strdup(conf->scriptFile);
free(base->scriptFile);
base->scriptFile = NULL;
vfsInit(base->container, base->dataDirBase, NULL);
luaL_openlibs(L);
if ((_luaLoadFile(L, "games.dat", NULL) != LUA_OK) || (lua_pcall(L, 0, 0, 0) != LUA_OK)) {
utilDie("%s: %s", base->container, lua_tostring(L, -1));
}
lua_getglobal(L, "GAMES");
if (!lua_istable(L, -1) || (lua_rawgeti(L, -1, 1) != LUA_TTABLE)) {
utilDie("%s: games.dat defines no GAMES entries.", base->container);
}
lua_replace(L, 1);
lua_settop(L, 1);
// --disc on the command line stands in for a DISC = true the entry forgot.
if (conf->disc) {
lua_pushboolean(L, true);
lua_setfield(L, 1, "DISC");
}
result = _buildConfFromTable(L, base);
if (result->scriptFile == NULL) {
utilDie("%s: the first games.dat entry has no SCRIPT.", base->container);
}
free(result->container);
result->container = strdup(base->container);
lua_close(L);
destroyConf(&base);
return result;
}
void singe(SDL_Window *window, SDL_Renderer *renderer, ConfigT *conf) {
int32_t x = 0;
int32_t y = 0;
@ -4362,6 +4522,7 @@ void singe(SDL_Window *window, SDL_Renderer *renderer, ConfigT *conf) {
// Local copy of config
_global.conf = cloneConf(conf);
vfsInit(_global.conf->container, _global.conf->dataDirBase, _global.conf->dataDir);
videoSetAudioDelay(_global.conf->audioDelayMs);
videoSetAudioCalibration(_loadAudioCalibration());
utilTrace("Audio delay: device queue %d ms, calibration %d ms, game %d ms", videoGetAudioLatency(), videoGetAudioCalibration(), videoGetAudioDelay());
@ -4584,8 +4745,6 @@ void singe(SDL_Window *window, SDL_Renderer *renderer, ConfigT *conf) {
// Open main video file, if this is a laserdisc game. Otherwise the canvas is the world.
if (_global.conf->disc) {
_progTrace("Opening main video file");
_doIndexDisplay(INDEX_DISPLAY_START);
videoSetIndexCallback(_doIndexDisplay);
if (_global.conf->isFrameFile) {
_global.frameFileHandle = frameFileLoad(_global.conf->videoFile, _global.conf->dataDir, _global.renderer, _global.conf->showCalculated);
frameFileSeek(_global.frameFileHandle, 0, &_global.videoHandle, &thisFrame); // Fills in _global.videoHandle
@ -4593,8 +4752,6 @@ void singe(SDL_Window *window, SDL_Renderer *renderer, ConfigT *conf) {
_global.videoHandle = videoLoad(_global.conf->videoFile, NULL, _global.conf->dataDir, _global.renderer, false);
}
videoSetVolume(_global.videoHandle, _global.conf->volumeVldp, _global.conf->volumeVldp);
videoSetIndexCallback(NULL);
_doIndexDisplay(INDEX_DISPLAY_STOP);
_global.canvasWidth = videoGetWidth(_global.videoHandle);
_global.canvasHeight = videoGetHeight(_global.videoHandle);
} else {
@ -4790,7 +4947,7 @@ void singe(SDL_Window *window, SDL_Renderer *renderer, ConfigT *conf) {
// Start script
_progTrace("Running %s", _global.conf->scriptFile);
lua_pushcfunction(_global.luaContext, _luaTraceback);
if (luaL_loadfile(_global.luaContext, _global.conf->scriptFile) || lua_pcall(_global.luaContext, 0, 0, -2)) {
if (_luaLoadFile(_global.luaContext, _global.conf->scriptFile, NULL) || lua_pcall(_global.luaContext, 0, 0, -2)) {
utilDie("Error running script: %s", lua_tostring(_global.luaContext, -1));
}
lua_settop(_global.luaContext, 0);
@ -4834,13 +4991,18 @@ void singe(SDL_Window *window, SDL_Renderer *renderer, ConfigT *conf) {
if (slot < 0) {
break;
}
if ((event.gbutton.button < CONTROLLER_BUTTON_COUNT) && _global.buttonSuppressed[slot][event.gbutton.button]) {
if (event.gbutton.button < CONTROLLER_BUTTON_COUNT) {
if ((event.type == SDL_EVENT_GAMEPAD_BUTTON_DOWN) && (SDL_GetTicks() < _global.inputGraceUntil)) {
_global.buttonSuppressed[slot][event.gbutton.button] = true;
}
if (_global.buttonSuppressed[slot][event.gbutton.button]) {
// Held since before this script: swallow it, and its release.
if (event.type == SDL_EVENT_GAMEPAD_BUTTON_UP) {
_global.buttonSuppressed[slot][event.gbutton.button] = false;
}
break;
}
}
code = CODE_GAMEPAD_BASE + slot * CODE_GAMEPAD_STRIDE + CODE_GAMEPAD_BUTTON_OFFSET + event.gbutton.button;
_processKey(event.type == SDL_EVENT_GAMEPAD_BUTTON_DOWN, 0, code);
break;
@ -4856,13 +5018,18 @@ void singe(SDL_Window *window, SDL_Renderer *renderer, ConfigT *conf) {
if (event.key.repeat && (_global.keyboardMode == KEYBOARD_NORMAL)) {
break;
}
if ((event.key.scancode < SDL_SCANCODE_COUNT) && _global.keySuppressed[event.key.scancode]) {
if (event.key.scancode < SDL_SCANCODE_COUNT) {
if ((event.type == SDL_EVENT_KEY_DOWN) && (SDL_GetTicks() < _global.inputGraceUntil)) {
_global.keySuppressed[event.key.scancode] = true;
}
if (_global.keySuppressed[event.key.scancode]) {
// Held since before this script: swallow it, and its release.
if (event.type == SDL_EVENT_KEY_UP) {
_global.keySuppressed[event.key.scancode] = false;
}
break;
}
}
_processKey(event.type == SDL_EVENT_KEY_DOWN, event.key.key, event.key.scancode);
break;

15
src/singe.h
View file

@ -25,7 +25,7 @@
#define SINGE_H
#include <SDL2/SDL.h>
#include <SDL3/SDL.h>
#include "common.h"
#include "generated/version.h"
@ -35,6 +35,13 @@
// Number of --sindengun arguments selects the border style.
typedef enum ToolModeE {
TOOL_NONE = 0,
TOOL_PACK,
TOOL_PATCH,
TOOL_UNPACK
} ToolModeE;
typedef enum SindenModeE {
SINDEN_WHITE = 1,
SINDEN_WHITE_BLACK = 2,
@ -47,6 +54,10 @@ typedef enum SindenModeE {
typedef struct ConfigS {
char *videoFile;
char *scriptFile;
char *container; // Game database holding the script, NULL for a loose game
char *toolSource; // Argument of --pack, --unpack, or --patch
ToolModeE toolMode;
bool dataDirGiven; // -d was on the command line
char *dataDir;
char *dataDirBase;
bool resolutionWasCalculated;
@ -64,6 +75,7 @@ typedef struct ConfigS {
bool scriptTracing;
bool legacySpriteArgs;
bool disc; // Play a laserdisc video; otherwise the canvas is the world
bool softwareVideo; // Skip the platform's hardware decoder
int32_t bestRatioIndex;
int32_t volumeVldp;
int32_t volumeNonVldp;
@ -79,6 +91,7 @@ typedef struct ConfigS {
} ConfigT;
ConfigT *confFromDatabase(const ConfigT *conf);
void singe(SDL_Window *window, SDL_Renderer *renderer, ConfigT *conf);

772
src/vfs.c Normal file
View file

@ -0,0 +1,772 @@
/*
* Singe virtual filesystem. See vfs.h for the lookup rules.
*
* Database layout (format version 1):
* meta(key TEXT PRIMARY KEY, value TEXT) version, gamedir, chunk, packer
* assets(path TEXT PRIMARY KEY, name TEXT, size INTEGER, data BLOB)
* chunks(path TEXT, chunk INTEGER, data BLOB, PRIMARY KEY(path, chunk))
* path is the normalised key (lower case, forward slashes); name keeps the
* author's spelling for unpacking. Files larger than meta.chunk live in
* chunks with assets.data NULL.
*/
#include <ctype.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <strings.h>
#include <sys/stat.h>
#include "sqlite3.h"
#include "util.h"
#include "vfs.h"
#ifdef _WIN32
#define fileSeek _fseeki64
#define fileTell _ftelli64
#else
#define fileSeek fseeko
#define fileTell ftello
#endif
#define CACHE_DIRECTORY "cache"
#define ENGINE_DIRECTORY "Singe"
#define OVERLAY_DIRECTORY "files"
#define META_CHUNK "chunk"
#define META_GAMEDIR "gamedir"
#define META_VERSION "version"
typedef struct DatabaseS {
char *path; // Database file as named by the caller
char *loose; // Sibling directory holding loose overrides, trailing separator
char *overlay; // Copy-on-write directory under the data directory, trailing separator
char *cache; // Where read-only io opens of packed assets are unpacked to; never searched
char *gameDir; // Lower case own-directory prefix the packer recorded, or NULL
int64_t chunkBytes;
sqlite3 *db;
sqlite3_stmt *assetStmt; // size, data by path
sqlite3_stmt *chunkStmt; // data by path and chunk number
struct DatabaseS *next;
} DatabaseT;
// Where a name lands. A NULL database means the plain filesystem path.
typedef struct TargetS {
DatabaseT *db;
char *path; // Filesystem path, or the loose override candidate for a packed name
char *key; // Normalised key inside the database
char *overlay; // Overlay path for the key
char *cache; // Cache path for the key
} TargetT;
struct VfsStreamS {
FILE *file; // Filesystem-backed stream, or NULL
DatabaseT *db;
char *key;
int64_t size;
int64_t position;
int64_t chunkIndex; // Chunk held in buffer, -1 for none
int64_t chunkLength;
uint8_t *buffer;
bool whole; // buffer holds the entire asset
};
static bool _assetExists(DatabaseT *db, const char *key);
static bool _assetSize(DatabaseT *db, const char *key, int64_t *size);
static DatabaseT *_databaseOpen(const char *path);
static bool _databaseReadMeta(DatabaseT *db, const char *key, char **value);
static bool _fileModified(const char *path, int64_t *size, int64_t *modified);
static bool _isAbsolute(const char *name);
static bool _isEngineName(const char *name);
static char *_normalise(const char *name);
static char *_overlayFor(const char *dataDirBase, const char *dataDir, const char *databasePath, bool isContainer, const char *directory);
static uint8_t *_readAsset(DatabaseT *db, const char *key, size_t *bytes, bool sdlMemory);
static bool _readChunk(DatabaseT *db, const char *key, int64_t index, uint8_t *buffer, int64_t *length);
static bool _resolve(const char *name, TargetT *target);
static void _targetFree(TargetT *target);
static bool _writeFile(const char *path, const uint8_t *data, size_t bytes);
static DatabaseT *_databases = NULL;
static DatabaseT *_container = NULL;
static char *_dataDirBase = NULL;
static char *_dataDir = NULL;
static bool _assetExists(DatabaseT *db, const char *key) {
int64_t size = 0;
return _assetSize(db, key, &size);
}
static bool _assetSize(DatabaseT *db, const char *key, int64_t *size) {
bool found = false;
sqlite3_reset(db->assetStmt);
sqlite3_bind_text(db->assetStmt, 1, key, -1, SQLITE_STATIC);
if (sqlite3_step(db->assetStmt) == SQLITE_ROW) {
*size = sqlite3_column_int64(db->assetStmt, 0);
found = true;
}
sqlite3_reset(db->assetStmt);
return found;
}
// Opens a game database read only, or returns the cached handle. NULL when the file is not one of ours.
static DatabaseT *_databaseOpen(const char *path) {
DatabaseT *db = NULL;
char *uri = NULL;
char *value = NULL;
char *encoded = NULL;
size_t i = 0;
size_t o = 0;
int64_t version = 0;
for (db = _databases; db != NULL; db = db->next) {
if (strcmp(db->path, path) == 0) {
return db;
}
}
// Percent-encode the three characters a URI would misread.
encoded = (char *)calloc(strlen(path) * 3 + 1, 1);
for (i = 0; path[i] != 0; i++) {
if ((path[i] == '?') || (path[i] == '#') || (path[i] == '%')) {
sprintf(encoded + o, "%%%02X", (unsigned char)path[i]);
o += 3;
} else {
encoded[o++] = path[i];
}
}
uri = utilCreateString("file:%s?immutable=1", encoded);
free(encoded);
db = (DatabaseT *)calloc(1, sizeof(DatabaseT));
db->path = strdup(path);
if (sqlite3_open_v2(uri, &db->db, SQLITE_OPEN_READONLY | SQLITE_OPEN_URI, NULL) != SQLITE_OK) {
free(uri);
sqlite3_close(db->db);
free(db->path);
free(db);
return NULL;
}
free(uri);
if (!_databaseReadMeta(db, META_VERSION, &value)) {
sqlite3_close(db->db);
free(db->path);
free(db);
return NULL;
}
version = strtoll(value, NULL, 10);
free(value);
if (version > VFS_FORMAT_VERSION) {
utilDie("%s is a format %lld game database; this Singe understands up to format %d.", path, (long long)version, VFS_FORMAT_VERSION);
}
if (sqlite3_prepare_v2(db->db, "SELECT size, data FROM assets WHERE path = ?", -1, &db->assetStmt, NULL) != SQLITE_OK) {
utilDie("%s has no assets table: %s", path, sqlite3_errmsg(db->db));
}
if (sqlite3_prepare_v2(db->db, "SELECT data FROM chunks WHERE path = ? AND chunk = ?", -1, &db->chunkStmt, NULL) != SQLITE_OK) {
utilDie("%s has no chunks table: %s", path, sqlite3_errmsg(db->db));
}
db->chunkBytes = VFS_CHUNK_BYTES;
if (_databaseReadMeta(db, META_CHUNK, &value)) {
db->chunkBytes = strtoll(value, NULL, 10);
free(value);
}
if (_databaseReadMeta(db, META_GAMEDIR, &value)) {
db->gameDir = _normalise(value);
free(value);
}
// Loose overrides live in a directory named like the database without its extension.
db->loose = utilCreateString("%.*s%c", (int)(strlen(path) - strlen(VFS_DATABASE_EXTENSION)), path, utilGetPathSeparator());
db->next = _databases;
_databases = db;
return db;
}
static bool _databaseReadMeta(DatabaseT *db, const char *key, char **value) {
sqlite3_stmt *stmt = NULL;
bool found = false;
*value = NULL;
if (sqlite3_prepare_v2(db->db, "SELECT value FROM meta WHERE key = ?", -1, &stmt, NULL) != SQLITE_OK) {
return false;
}
sqlite3_bind_text(stmt, 1, key, -1, SQLITE_STATIC);
if ((sqlite3_step(stmt) == SQLITE_ROW) && (sqlite3_column_text(stmt, 0) != NULL)) {
*value = strdup((const char *)sqlite3_column_text(stmt, 0));
found = true;
}
sqlite3_finalize(stmt);
return found;
}
static bool _fileModified(const char *path, int64_t *size, int64_t *modified) {
struct stat info;
if (stat(path, &info) != 0) {
return false;
}
*size = (int64_t)info.st_size;
*modified = (int64_t)info.st_mtime;
return true;
}
static bool _isAbsolute(const char *name) {
return (name[0] == '/') || (isalpha((unsigned char)name[0]) && (name[1] == ':'));
}
// Names the engine owns resolve on the filesystem even inside a packed game: Singe/ and the data directory.
static bool _isEngineName(const char *name) {
size_t length = strlen(ENGINE_DIRECTORY);
char *base = NULL;
bool engine = false;
if ((utilStricmp(name, ENGINE_DIRECTORY) == 0) || ((strncmp(name, ENGINE_DIRECTORY, length) == 0) && (name[length] == '/'))) {
return true;
}
if (_dataDirBase != NULL) {
base = _normalise(_dataDirBase);
engine = (strncmp(name, base, strlen(base)) == 0);
free(base);
}
return engine;
}
// Forward slashes, no leading "./", no doubled slashes. Case is untouched.
static char *_normalise(const char *name) {
char *out = strdup(name);
size_t i = 0;
size_t o = 0;
for (i = 0; out[i] != 0; i++) {
char c = (out[i] == '\\') ? '/' : out[i];
if ((c == '/') && (o > 0) && (out[o - 1] == '/')) {
continue;
}
if ((c == '.') && (out[i + 1] == '/' || out[i + 1] == '\\') && ((o == 0) || (out[o - 1] == '/'))) {
i++;
continue;
}
out[o++] = c;
}
out[o] = 0;
return out;
}
// A per-game directory under the data directory: the overlay or the cache.
static char *_overlayFor(const char *dataDirBase, const char *dataDir, const char *databasePath, bool isContainer, const char *directory) {
const char *base = NULL;
size_t length = 0;
if (isContainer && (dataDir != NULL)) {
return utilCreateString("%s%s%c", dataDir, directory, utilGetPathSeparator());
}
base = utilGetLastPathComponent(databasePath);
length = strlen(base) - strlen(VFS_DATABASE_EXTENSION);
return utilCreateString("%s%.*s%c%s%c", dataDirBase ? dataDirBase : "", (int)length, base, utilGetPathSeparator(), directory, utilGetPathSeparator());
}
// Whole asset, from the row or reassembled from its chunks.
static uint8_t *_readAsset(DatabaseT *db, const char *key, size_t *bytes, bool sdlMemory) {
uint8_t *data = NULL;
int64_t size = 0;
int64_t offset = 0;
int64_t index = 0;
int64_t length = 0;
*bytes = 0;
sqlite3_reset(db->assetStmt);
sqlite3_bind_text(db->assetStmt, 1, key, -1, SQLITE_STATIC);
if (sqlite3_step(db->assetStmt) != SQLITE_ROW) {
sqlite3_reset(db->assetStmt);
return NULL;
}
size = sqlite3_column_int64(db->assetStmt, 0);
data = sdlMemory ? (uint8_t *)SDL_malloc((size_t)size + 1) : (uint8_t *)malloc((size_t)size + 1);
if (data == NULL) {
utilDie("Out of memory reading %s from %s.", key, db->path);
}
if (sqlite3_column_type(db->assetStmt, 1) != SQLITE_NULL) {
memcpy(data, sqlite3_column_blob(db->assetStmt, 1), (size_t)sqlite3_column_bytes(db->assetStmt, 1));
offset = sqlite3_column_bytes(db->assetStmt, 1);
}
sqlite3_reset(db->assetStmt);
while (offset < size) {
if (!_readChunk(db, key, index, data + offset, &length) || (length == 0)) {
utilDie("%s in %s is missing chunk %lld.", key, db->path, (long long)index);
}
offset += length;
index++;
}
data[size] = 0;
*bytes = (size_t)size;
return data;
}
static bool _readChunk(DatabaseT *db, const char *key, int64_t index, uint8_t *buffer, int64_t *length) {
bool found = false;
*length = 0;
sqlite3_reset(db->chunkStmt);
sqlite3_bind_text(db->chunkStmt, 1, key, -1, SQLITE_STATIC);
sqlite3_bind_int64(db->chunkStmt, 2, index);
if (sqlite3_step(db->chunkStmt) == SQLITE_ROW) {
*length = sqlite3_column_bytes(db->chunkStmt, 0);
if (*length > db->chunkBytes) {
utilDie("%s in %s has an oversized chunk.", key, db->path);
}
memcpy(buffer, sqlite3_column_blob(db->chunkStmt, 0), (size_t)*length);
found = true;
}
sqlite3_reset(db->chunkStmt);
return found;
}
// Decides where a name lives. Returns false only for an empty name.
static bool _resolve(const char *name, TargetT *target) {
char *norm = _normalise(name);
char *prefix = NULL;
const char *inner = NULL;
char *p = NULL;
size_t i = 0;
size_t extLen = strlen(VFS_DATABASE_EXTENSION);
memset(target, 0, sizeof(*target));
if (norm[0] == 0) {
free(norm);
return false;
}
// A path that passes through a database file addresses its contents: "Games/DLe.game/Overlay/x.png".
if (!_isAbsolute(norm)) {
for (i = extLen; norm[i] != 0; i++) {
if ((norm[i] == '/') && (strncasecmp(norm + i - extLen, VFS_DATABASE_EXTENSION, extLen) == 0)) {
prefix = utilStrndup(norm, i);
if (utilFileExists(prefix)) {
target->db = _databaseOpen(prefix);
if (target->db != NULL) {
inner = norm + i + 1;
if (target->db->overlay == NULL) {
target->db->overlay = _overlayFor(_dataDirBase, _dataDir, prefix, false, OVERLAY_DIRECTORY);
target->db->cache = _overlayFor(_dataDirBase, _dataDir, prefix, false, CACHE_DIRECTORY);
}
free(prefix);
break;
}
}
free(prefix);
prefix = NULL;
}
}
}
if ((target->db == NULL) && (_container != NULL) && !_isAbsolute(norm) && !_isEngineName(norm)) {
target->db = _container;
inner = norm;
}
if (target->db == NULL) {
target->path = strdup(name);
free(norm);
return true;
}
// Inside a database: drop the game's own directory when the name starts with it.
if ((target->db->gameDir != NULL) && (inner[0] != 0)) {
i = strlen(target->db->gameDir);
if ((utilStricmp(inner, target->db->gameDir) == 0) || ((strncasecmp(inner, target->db->gameDir, i) == 0) && (inner[i] == '/'))) {
inner += i;
while (*inner == '/') {
inner++;
}
}
}
target->key = strdup(inner);
for (p = target->key; *p != 0; p++) {
*p = (char)tolower((unsigned char)*p);
}
target->path = utilCreateString("%s%s", target->db->loose, inner);
target->overlay = utilCreateString("%s%s", target->db->overlay, target->key);
target->cache = utilCreateString("%s%s", target->db->cache, target->key);
utilFixPathSeparators(&target->path, false);
utilFixPathSeparators(&target->overlay, false);
utilFixPathSeparators(&target->cache, false);
free(norm);
return true;
}
static void _targetFree(TargetT *target) {
free(target->path);
free(target->key);
free(target->overlay);
free(target->cache);
memset(target, 0, sizeof(*target));
}
static bool _writeFile(const char *path, const uint8_t *data, size_t bytes) {
char *directory = utilGetUpToLastPathComponent(path);
FILE *file = NULL;
bool ok = false;
if ((directory[0] == 0) || utilMkDirP(directory, 0755)) {
file = fopen(path, "wb");
if (file != NULL) {
ok = (fwrite(data, 1, bytes, file) == bytes);
fclose(file);
}
}
free(directory);
return ok;
}
bool vfsExists(const char *name) {
TargetT target;
bool found = false;
if (!_resolve(name, &target)) {
return false;
}
found = utilFileExists(target.path);
if (!found && (target.db != NULL)) {
found = utilFileExists(target.overlay) || _assetExists(target.db, target.key);
}
_targetFree(&target);
return found;
}
// A filesystem path Lua's io library can open for the name. Writes land in the loose directory or
// the overlay, never the database. A packed asset opened for reading is unpacked into the cache,
// which the lookup never searches, so a later patch of the asset is not shadowed by the copy.
char *vfsFilePath(const char *name, bool forWriting) {
TargetT target;
char *path = NULL;
uint8_t *data = NULL;
size_t bytes = 0;
if (!_resolve(name, &target)) {
return strdup(name);
}
if (target.db == NULL) {
path = target.path;
target.path = NULL;
} else if (utilFileExists(target.path)) {
path = target.path;
target.path = NULL;
} else if (forWriting || utilFileExists(target.overlay)) {
if (forWriting) {
char *directory = utilGetUpToLastPathComponent(target.overlay);
utilMkDirP(directory, 0755);
free(directory);
}
path = target.overlay;
target.overlay = NULL;
} else {
data = _readAsset(target.db, target.key, &bytes, false);
if (data != NULL) {
if (!_writeFile(target.cache, data, bytes)) {
utilDie("Unable to copy %s to %s.", name, target.cache);
}
free(data);
path = target.cache;
target.cache = NULL;
} else {
path = strdup(name);
}
}
_targetFree(&target);
return path;
}
// container is the game's database, or NULL for a loose game. dataDir carries a trailing separator.
void vfsInit(const char *container, const char *dataDirBase, const char *dataDir) {
free(_dataDirBase);
free(_dataDir);
_dataDirBase = dataDirBase ? strdup(dataDirBase) : NULL;
_dataDir = dataDir ? strdup(dataDir) : NULL;
_container = NULL;
if (container != NULL) {
_container = _databaseOpen(container);
if (_container == NULL) {
utilDie("%s is not a Singe game database.", container);
}
free(_container->overlay);
free(_container->cache);
_container->overlay = _overlayFor(_dataDirBase, _dataDir, container, true, OVERLAY_DIRECTORY);
_container->cache = _overlayFor(_dataDirBase, _dataDir, container, true, CACHE_DIRECTORY);
}
}
bool vfsIsDatabase(const char *path) {
size_t length = strlen(path);
size_t extLen = strlen(VFS_DATABASE_EXTENSION);
if ((length <= extLen) || (utilStricmp(path + length - extLen, VFS_DATABASE_EXTENSION) != 0) || !utilFileExists(path)) {
return false;
}
return _databaseOpen(path) != NULL;
}
// Read-only stream for the SDL loaders. Packed assets are served from memory.
SDL_IOStream *vfsOpenIO(const char *name) {
TargetT target;
SDL_IOStream *io = NULL;
uint8_t *data = NULL;
size_t bytes = 0;
if (!_resolve(name, &target)) {
return NULL;
}
if ((target.db == NULL) || utilFileExists(target.path)) {
io = SDL_IOFromFile(target.path, "rb");
} else if (utilFileExists(target.overlay)) {
io = SDL_IOFromFile(target.overlay, "rb");
} else {
data = _readAsset(target.db, target.key, &bytes, true);
if (data != NULL) {
io = SDL_IOFromMem(data, bytes);
if (io == NULL) {
SDL_free(data);
} else {
SDL_SetPointerProperty(SDL_GetIOProperties(io), SDL_PROP_IOSTREAM_MEMORY_FREE_FUNC_POINTER, SDL_free);
}
} else {
SDL_SetError("%s not found", name);
}
}
_targetFree(&target);
return io;
}
void vfsQuit(void) {
DatabaseT *db = NULL;
DatabaseT *next = NULL;
for (db = _databases; db != NULL; db = next) {
next = db->next;
sqlite3_finalize(db->assetStmt);
sqlite3_finalize(db->chunkStmt);
sqlite3_close(db->db);
free(db->path);
free(db->loose);
free(db->overlay);
free(db->cache);
free(db->gameDir);
free(db);
}
_databases = NULL;
_container = NULL;
free(_dataDirBase);
free(_dataDir);
_dataDirBase = NULL;
_dataDir = NULL;
}
// Whole file, NUL terminated, free() when done. NULL when the name resolves nowhere.
char *vfsRead(const char *name, size_t *bytes) {
TargetT target;
char *data = NULL;
*bytes = 0;
if (!_resolve(name, &target)) {
return NULL;
}
if ((target.db == NULL) || utilFileExists(target.path)) {
data = utilReadFile(target.path, bytes);
} else if (utilFileExists(target.overlay)) {
data = utilReadFile(target.overlay, bytes);
} else {
data = (char *)_readAsset(target.db, target.key, bytes, false);
}
_targetFree(&target);
return data;
}
// Size and modification time, the latter being the database's own for a packed asset.
bool vfsStat(const char *name, int64_t *size, int64_t *modified) {
TargetT target;
bool found = false;
int64_t ignore = 0;
if (!_resolve(name, &target)) {
return false;
}
found = _fileModified(target.path, size, modified);
if (!found && (target.db != NULL)) {
found = _fileModified(target.overlay, size, modified);
if (!found && _assetSize(target.db, target.key, size)) {
found = _fileModified(target.db->path, &ignore, modified);
}
}
_targetFree(&target);
return found;
}
void vfsStreamClose(VfsStreamT *stream) {
if (stream == NULL) {
return;
}
if (stream->file != NULL) {
fclose(stream->file);
}
free(stream->key);
free(stream->buffer);
free(stream);
}
// Seekable byte stream for the video demuxer. Chunked assets are read one chunk at a time.
VfsStreamT *vfsStreamOpen(const char *name) {
TargetT target;
VfsStreamT *stream = NULL;
const char *path = NULL;
size_t bytes = 0;
if (!_resolve(name, &target)) {
return NULL;
}
stream = (VfsStreamT *)calloc(1, sizeof(VfsStreamT));
stream->chunkIndex = -1;
if ((target.db == NULL) || utilFileExists(target.path)) {
path = target.path;
} else if (utilFileExists(target.overlay)) {
path = target.overlay;
}
if (path != NULL) {
stream->file = fopen(path, "rb");
if (stream->file == NULL) {
free(stream);
stream = NULL;
} else {
fileSeek(stream->file, 0, SEEK_END);
stream->size = (int64_t)fileTell(stream->file);
fileSeek(stream->file, 0, SEEK_SET);
}
} else if (_assetSize(target.db, target.key, &stream->size)) {
stream->db = target.db;
stream->key = strdup(target.key);
if (stream->size <= target.db->chunkBytes) {
// Small enough to hold whole; the row carries the data.
stream->buffer = _readAsset(target.db, target.key, &bytes, false);
stream->chunkIndex = 0;
stream->chunkLength = (int64_t)bytes;
stream->whole = true;
} else {
stream->buffer = (uint8_t *)malloc((size_t)target.db->chunkBytes);
}
} else {
free(stream);
stream = NULL;
}
_targetFree(&target);
return stream;
}
int64_t vfsStreamRead(VfsStreamT *stream, void *buffer, int64_t bytes) {
uint8_t *out = (uint8_t *)buffer;
int64_t total = 0;
int64_t index = 0;
int64_t start = 0;
int64_t take = 0;
if (stream->file != NULL) {
total = (int64_t)fread(buffer, 1, (size_t)bytes, stream->file);
stream->position += total;
return total;
}
if (bytes > stream->size - stream->position) {
bytes = stream->size - stream->position;
}
while (total < bytes) {
index = stream->whole ? 0 : stream->position / stream->db->chunkBytes;
start = stream->whole ? stream->position : stream->position % stream->db->chunkBytes;
if (index != stream->chunkIndex) {
if (!_readChunk(stream->db, stream->key, index, stream->buffer, &stream->chunkLength)) {
break;
}
stream->chunkIndex = index;
}
take = stream->chunkLength - start;
if (take > bytes - total) {
take = bytes - total;
}
if (take <= 0) {
break;
}
memcpy(out + total, stream->buffer + start, (size_t)take);
total += take;
stream->position += take;
}
return total;
}
int64_t vfsStreamSeek(VfsStreamT *stream, int64_t offset, int32_t whence) {
int64_t position = offset;
if (whence == SEEK_CUR) {
position = stream->position + offset;
} else if (whence == SEEK_END) {
position = stream->size + offset;
}
if ((position < 0) || (position > stream->size)) {
return -1;
}
if ((stream->file != NULL) && (fileSeek(stream->file, position, SEEK_SET) != 0)) {
return -1;
}
stream->position = position;
return position;
}
int64_t vfsStreamSize(VfsStreamT *stream) {
return stream->size;
}

42
src/vfs.h Normal file
View file

@ -0,0 +1,42 @@
/*
* Singe virtual filesystem: one lookup for every file a game names.
*
* A name resolves to a loose file in the game directory, to a copy-on-write
* overlay in the data directory, or to an asset inside the game's SQLite
* database, in that order. Loose games with no database see exactly the
* filesystem they always did.
*/
#ifndef VFS_H
#define VFS_H
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
#include <SDL3/SDL.h>
#define VFS_DATABASE_EXTENSION ".game"
#define VFS_FORMAT_VERSION 1
#define VFS_CHUNK_BYTES (4 * 1024 * 1024)
typedef struct VfsStreamS VfsStreamT;
bool vfsExists(const char *name);
char *vfsFilePath(const char *name, bool forWriting);
void vfsInit(const char *container, const char *dataDirBase, const char *dataDir);
bool vfsIsDatabase(const char *path);
SDL_IOStream *vfsOpenIO(const char *name);
void vfsQuit(void);
char *vfsRead(const char *name, size_t *bytes);
bool vfsStat(const char *name, int64_t *size, int64_t *modified);
void vfsStreamClose(VfsStreamT *stream);
VfsStreamT *vfsStreamOpen(const char *name);
int64_t vfsStreamRead(VfsStreamT *stream, void *buffer, int64_t bytes);
int64_t vfsStreamSeek(VfsStreamT *stream, int64_t offset, int32_t whence);
int64_t vfsStreamSize(VfsStreamT *stream);
#endif

1263
src/videoPlayer.c
View file

File diff suppressed because it is too large Load diff

5
src/videoPlayer.h
View file

@ -35,9 +35,6 @@
#define VIDEO_AUDIO_DELAY_MAX 1000 // Milliseconds either way
typedef void (*VideoIndexingCallbackT)(int32_t percent);
int32_t videoGetAudioCalibration(void);
int32_t videoGetAudioDelay(void);
int32_t videoGetAudioLatency(void);
@ -64,7 +61,7 @@ void videoSeek(int32_t playerHandle, int64_t seekFrame);
void videoSetAudioCalibration(int32_t milliseconds);
void videoSetAudioDelay(int32_t milliseconds);
void videoSetAudioTrack(int32_t playerHandle, int32_t track);
void videoSetIndexCallback(VideoIndexingCallbackT callback);
void videoSetHardwareDecoding(bool enabled);
void videoSetVolume(int32_t playerHandle, int32_t leftPercent, int32_t rightPercent);
void videoUnlockAudio(void);
void videoUnload(int32_t playerHandle);

83
thirdparty/bzip2/.appveyor.yml vendored
View file

@ -1,83 +0,0 @@
environment:
matrix:
- COMPILER: MSVC2019
PLATFORM: x86
CC: cl.exe
APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2019
CMAKE_GENERATOR: NMake Makefiles
- COMPILER: MSVC2019
PLATFORM: amd64
CC: cl.exe
APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2019
CMAKE_GENERATOR: NMake Makefiles
- COMPILER: MSVC2017
PLATFORM: x86
CC: cl.exe
APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2017
CMAKE_GENERATOR: NMake Makefiles
- COMPILER: MSVC2017
PLATFORM: amd64
CC: cl.exe
APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2017
CMAKE_GENERATOR: NMake Makefiles
- COMPILER: MSVC2015
PLATFORM: x86
CC: cl.exe
APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2015
CMAKE_GENERATOR: NMake Makefiles
- COMPILER: MSVC2015
PLATFORM: amd64
CC: cl.exe
APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2015
CMAKE_GENERATOR: NMake Makefiles
- COMPILER: GCC
ARCH: 32
CC: gcc.exe
CMAKE_GENERATOR: MinGW Makefiles
- COMPILER: GCC
ARCH: 64
CC: gcc.exe
CMAKE_GENERATOR: MinGW Makefiles
install:
- if [%COMPILER%]==[MSVC2015] if [%PLATFORM%]==[amd64] call "C:\Program Files\Microsoft SDKs\Windows\v7.1\Bin\SetEnv.cmd" /x64
- if [%COMPILER%]==[MSVC2015] if [%PLATFORM%]==[amd64] call "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall.bat" x86_amd64
- if [%COMPILER%]==[MSVC2015] if [%PLATFORM%]==[x86] call "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall.bat" x86
- if [%COMPILER%]==[MSVC2017] if [%PLATFORM%]==[amd64] call "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Auxiliary\Build\vcvars64.bat"
- if [%COMPILER%]==[MSVC2017] if [%PLATFORM%]==[x86] call "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Auxiliary\Build\vcvars32.bat"
- if [%COMPILER%]==[MSVC2019] call "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvarsall.bat" %PLATFORM%
- if [%COMPILER%]==[GCC] del /F /Q "C:\Program Files\Git\usr\bin\sh.exe"
- if [%COMPILER%]==[GCC] set "PATH=C:\msys64\mingw%ARCH%\bin;%PATH%"
build_script:
# Clone the files for the large test suite
# FIXME: git submodule isn't working with the MinGW environment:
# fatal: 'submodule' appears to be a git command, but we were not able to execute it. Maybe git-submodule is broken?
- if not [%COMPILER%]==[GCC] git submodule update --init --recursive
- mkdir build
- cd build
- cmake -G "%CMAKE_GENERATOR%" -DCMAKE_BUILD_TYPE=Release ..
- cmake --build . --target install
test_script:
- ctest -V -C Release
# Enable this to be able to login to the build worker. You can use the
# `remmina` program in Ubuntu, use the login information that the line below
# prints into the log.
#on_finish:
#- ps: $blockRdp = $true; iex ((new-object net.webclient).DownloadString('https://raw.githubusercontent.com/appveyor/ci/master/scripts/enable-rdp.ps1'))

16
thirdparty/bzip2/.gitattributes vendored
View file

@ -1,16 +0,0 @@
# Handle line endings automatically for files detected as text
# and leave all files detected as binary untouched.
* text=auto
#
# The above will handle all files NOT found below
#
# Files that should be left untouched (binary is macro for -text -diff)
*.ref binary
#
# Exclude files from exporting
#
.gitattributes export-ignore
.gitignore export-ignore

32
thirdparty/bzip2/.gitignore vendored
View file

@ -1,32 +0,0 @@
# cmake
CMakeCache.txt
CMakeFiles/
cmake_install.cmake
install_manifest.txt
CTestTestfile.cmake
build.ninja
rules.ninja
.ninja_deps
.ninja_log
lib*.dylib
lib*.so
lib*.so.*
lib*.a
# example build directories
build
builddir
# vscode config directory
.vscode
# Windows build outputs
*.dll
*.exe
*.exp
*.lib
*.obj
*.res
# python unit tests
__pycache__

204
thirdparty/bzip2/.gitlab-ci.yml vendored
View file

@ -1,204 +0,0 @@
# CI setup based on https://gitlab.gnome.org/GNOME/librsvg/blob/master/.gitlab-ci.yml
variables:
AMD64_DEBIAN_TESTING: debian:testing
AMD64_UBUNTU_BIONIC: ubuntu:bionic
AMD64_FEDORA_LATEST: "registry.fedoraproject.org/fedora:35"
AMD64_FEDORA_RAWHIDE: "registry.fedoraproject.org/fedora:rawhide"
AMD64_OPENSUSE_LEAP: opensuse/leap
AMD64_OPENSUSE_TUMBLEWEED: opensuse/tumbleweed
I386_UBUNTU_BIONIC: "i386/ubuntu:bionic"
GIT_SUBMODULE_STRATEGY: recursive
stages:
- test
.meson_test:
stage: test
script:
- meson builddir
- ninja -C builddir
- meson test -C builddir --print-errorlogs
after_script:
- cp builddir/meson-logs/meson-log.txt .
- rm -rf builddir
artifacts:
paths:
- "meson-log.txt"
.cmake_test_debug:
stage: test
script:
- mkdir builddir && cd builddir
- cmake .. -DCMAKE_BUILD_TYPE="Debug" -D ENABLE_SHARED_LIB=ON -D ENABLE_STATIC_LIB=OFF
- cmake --build .
- make check
after_script:
- cp builddir/CMakeFiles/CMakeOutput.log .
- rm -rf builddir
artifacts:
paths:
- "CMakeOutput.log"
.cmake_test:
stage: test
script:
- mkdir builddir && cd builddir
- cmake .. -DCMAKE_BUILD_TYPE="Release" -D ENABLE_SHARED_LIB=ON -D ENABLE_STATIC_LIB=OFF
- cmake --build .
- make check
after_script:
- cp builddir/CMakeFiles/CMakeOutput.log .
- rm -rf builddir
artifacts:
paths:
- "CMakeOutput.log"
.cmake_test_shared_static:
stage: test
script:
- mkdir builddir && cd builddir
- cmake .. -DCMAKE_BUILD_TYPE="Release" -D ENABLE_SHARED_LIB=ON -D ENABLE_STATIC_LIB=ON
- cmake --build .
- make check
after_script:
- cp builddir/CMakeFiles/CMakeOutput.log .
- rm -rf builddir
artifacts:
paths:
- "CMakeOutput.log"
.cmake_test_static_only:
stage: test
script:
- mkdir builddir && cd builddir
- cmake .. -DCMAKE_BUILD_TYPE="Release" -D ENABLE_SHARED_LIB=OFF -D ENABLE_STATIC_LIB=ON
- cmake --build .
- make check
after_script:
- cp builddir/CMakeFiles/CMakeOutput.log .
- rm -rf builddir
artifacts:
paths:
- "CMakeOutput.log"
debian:testing:meson:
extends: ".meson_test"
image: $AMD64_DEBIAN_TESTING
before_script:
- apt-get update -y
- apt-get install -y gcc meson python3-distutils python3-pytest valgrind
debian:testing:cmake:debug:
extends: ".cmake_test_debug"
image: $AMD64_DEBIAN_TESTING
before_script:
- apt-get update -y
- apt-get install -y gcc make python3-pytest cmake valgrind
debian:testing:cmake:
extends: ".cmake_test"
image: $AMD64_DEBIAN_TESTING
before_script:
- apt-get update -y
- apt-get install -y gcc make python3-pytest cmake valgrind
debian:testing:cmake:shared_static:
extends: ".cmake_test_shared_static"
image: $AMD64_DEBIAN_TESTING
before_script:
- apt-get update -y
- apt-get install -y gcc make python3-pytest cmake valgrind
debian:testing:cmake:static_only:
extends: ".cmake_test_static_only"
image: $AMD64_DEBIAN_TESTING
before_script:
- apt-get update -y
- apt-get install -y gcc make python3-pytest cmake valgrind
ubuntu:bionic:meson:
extends: ".meson_test"
image: $AMD64_UBUNTU_BIONIC
before_script:
- apt-get update -y
- apt-get install -y gcc python3-pip ninja-build valgrind
- pip3 install meson pytest
ubuntu:bionic:cmake:
extends: ".cmake_test"
image: $AMD64_UBUNTU_BIONIC
before_script:
- apt-get update -y
- apt-get install -y gcc make python3-pip valgrind
- pip3 install --upgrade pip
- pip3 install pytest cmake
ubuntu:bionic:i386:meson:
extends: ".meson_test"
image: $I386_UBUNTU_BIONIC
before_script:
- apt-get update -y
- apt-get install -y gcc python3-pip ninja-build valgrind
- pip3 install meson pytest
ubuntu:bionic:i386:cmake:
extends: ".cmake_test"
image: $I386_UBUNTU_BIONIC
before_script:
- apt-get update -y
- apt-get install -y gcc make python3-pip valgrind
- pip3 install --upgrade pip
- pip3 install pytest cmake
fedora:30:meson:
extends: ".meson_test"
image: $AMD64_FEDORA_LATEST
before_script:
- dnf install -y gcc meson python3-pytest valgrind
fedora:30:cmake:
extends: ".cmake_test"
image: $AMD64_FEDORA_LATEST
before_script:
- dnf install -y gcc cmake python3-pytest valgrind
fedora:rawhide:meson:
extends: ".meson_test"
image: $AMD64_FEDORA_RAWHIDE
before_script:
- dnf install -y gcc meson python3-pytest valgrind
fedora:rawhide:cmake:
extends: ".cmake_test"
image: $AMD64_FEDORA_RAWHIDE
before_script:
- dnf install -y gcc cmake python3-pytest valgrind
opensuse/leap:meson:
extends: ".meson_test"
image: $AMD64_OPENSUSE_LEAP
before_script:
- zypper install -y gcc ninja python3-pip valgrind
- pip3 install meson pytest
opensuse/leap:cmake:
extends: ".cmake_test"
image: $AMD64_OPENSUSE_LEAP
before_script:
- zypper install -y gcc cmake python3-pip valgrind
- pip3 install pytest
opensuse/tumbleweed:meson:
extends: ".meson_test"
image: $AMD64_OPENSUSE_TUMBLEWEED
before_script:
- zypper install -y gcc meson python3-pytest valgrind
opensuse/tumbleweed:cmake:
extends: ".cmake_test"
image: $AMD64_OPENSUSE_TUMBLEWEED
before_script:
- zypper install -y gcc cmake python3-pytest valgrind

19
thirdparty/bzip2/AUTHORS vendored
View file

@ -1,19 +0,0 @@
Alson van der Meulen
Bo Lindbergh
Bo Thorsen
Jeremy Fusco
Joerg Prante
Jorj Bauer
Juan Pedro Vallejo
Julian Seward
Leland Lucius
Marty Leisner
Matthias Krings
Michael Carmack
Mikolaj Izdebski
Nicholas Nethercote
Rich Ireland
Robert Linden
Solar Designer
Trond Eivind Glomsrod
Volker Schmidt

472
thirdparty/bzip2/CMakeLists.txt vendored
View file

@ -1,472 +0,0 @@
cmake_minimum_required(VERSION 3.12)
project(bzip2
VERSION 1.1.0
DESCRIPTION "This Bzip2/libbz2 a program and library for lossless block-sorting data compression."
LANGUAGES C)
# See versioning rule:
# http://www.gnu.org/software/libtool/manual/html_node/Updating-version-info.html
#
# KEEP THESE IN SYNC WITH meson.build OR STUFF WILL BREAK!
set(LT_CURRENT 1)
set(LT_REVISION 9)
set(LT_AGE 0)
set(CMAKE_MODULE_PATH ${CMAKE_CURRENT_SOURCE_DIR}/cmake ${CMAKE_MODULE_PATH})
include(Version)
include(SymLink)
set(BZ_VERSION ${PROJECT_VERSION})
configure_file (
${PROJECT_SOURCE_DIR}/bz_version.h.in
${PROJECT_BINARY_DIR}/bz_version.h
)
include_directories(${PROJECT_BINARY_DIR})
math(EXPR LT_SOVERSION "${LT_CURRENT} - ${LT_AGE}")
set(LT_VERSION "${LT_SOVERSION}.${LT_AGE}.${LT_REVISION}")
set(PACKAGE_VERSION ${PROJECT_VERSION})
HexVersion(PACKAGE_VERSION_NUM ${PROJECT_VERSION_MAJOR} ${PROJECT_VERSION_MINOR} ${PROJECT_VERSION_PATCH})
set(ENABLE_APP_DEFAULT ON)
set(ENABLE_TESTS_DEFAULT ON)
set(ENABLE_EXAMPLES_DEFAULT OFF)
set(ENABLE_DOCS_DEFAULT OFF)
include(CMakeOptions.txt)
if(ENABLE_LIB_ONLY AND (ENABLE_APP OR ENABLE_EXAMPLES))
# Remember when disabled options are disabled for later diagnostics.
set(ENABLE_LIB_ONLY_DISABLED_OTHERS 1)
else()
set(ENABLE_LIB_ONLY_DISABLED_OTHERS 0)
endif()
if(ENABLE_LIB_ONLY)
set(ENABLE_APP OFF)
set(ENABLE_EXAMPLES OFF)
endif()
# Do not disable assertions based on CMAKE_BUILD_TYPE.
foreach(_build_type Release MinSizeRel RelWithDebInfo)
foreach(_lang C)
string(TOUPPER CMAKE_${_lang}_FLAGS_${_build_type} _var)
string(REGEX REPLACE "(^|)[/-]D *NDEBUG($|)" " " ${_var} "${${_var}}")
endforeach()
endforeach()
# Support the latest c++ standard available.
include(ExtractValidFlags)
if(NOT CMAKE_BUILD_TYPE AND NOT CMAKE_CONFIGURATION_TYPES)
set(CMAKE_BUILD_TYPE RelWithDebInfo CACHE STRING "Choose the build type" FORCE)
# Include "None" as option to disable any additional (optimization) flags,
# relying on just CMAKE_C_FLAGS and CMAKE_CXX_FLAGS (which are empty by
# default). These strings are presented in cmake-gui.
set_property(CACHE CMAKE_BUILD_TYPE PROPERTY STRINGS
None Debug Release MinSizeRel RelWithDebInfo)
endif()
include(GNUInstallDirs)
if(ENABLE_TESTS OR ENABLE_DOCS)
# For test scripts and documentation
find_package(Python3 REQUIRED)
endif()
#
# Find other Test dependencies
# - pytest (optional)
# - unittest (if pytest not present)
# - valgrind (optional, Linux only)
#
if(ENABLE_TESTS)
# Try finding pytest from the PATH
execute_process(
COMMAND pytest --version
RESULT_VARIABLE PYTEST_EXIT_CODE
ERROR_QUIET OUTPUT_QUIET
)
if(${PYTEST_EXIT_CODE} EQUAL 0)
# pytest found in the path.
set(PythonTest_COMMAND "pytest;-v")
else()
# Not in the path, try using: python3 -m pytest
execute_process(
COMMAND ${Python3_EXECUTABLE} -m pytest --version
RESULT_VARIABLE PYTEST_MODULE_EXIT_CODE
ERROR_QUIET OUTPUT_QUIET
)
if(${PYTEST_MODULE_EXIT_CODE} EQUAL 0)
# pytest isn't in the path, but the Python 3 we found has it.
set(PythonTest_COMMAND "${Python3_EXECUTABLE};-m;pytest;-v")
else()
# pytest couldn't be found, verify that we can at least use: python3 -m unittest
execute_process(
COMMAND ${Python3_EXECUTABLE} -m unittest --help
RESULT_VARIABLE UNITTEST_MODULE_EXIT_CODE
ERROR_QUIET OUTPUT_QUIET
)
if(${UNITTEST_MODULE_EXIT_CODE} EQUAL 0)
# No pytest :-(, but we'll get by with unittest
message("Python 3 package 'pytest' is not installed for ${Python3_EXECUTABLE} and is not available in your PATH.")
message("Failed unit tests will be easier to read if you install pytest.")
message("Eg: python3 -m pip install --user pytest")
set(PythonTest_COMMAND "${Python3_EXECUTABLE};-m;unittest;--verbose")
else()
# No unittest either!
# Some weird Python installations do exist that lack standard modules like unittest.
# Let's make sure these folks know the Python 3 install we found won't cut it.
message("Python 3 found: ${Python3_EXECUTABLE}, but it is missing the unittest module (wierd!).")
message(FATAL_ERROR "The tests won't work with this Python installation. You can disable the tests by reconfiguring with: -D ENABLE_TESTS=OFF")
endif()
endif()
endif()
# Check for valgrind. If it exists, we'll enable extra tests that use valgrind.
if(CMAKE_SYSTEM_NAME STREQUAL "Linux")
find_package(Valgrind)
endif()
endif()
# Checks for header files.
include(CheckIncludeFile)
check_include_file(arpa/inet.h HAVE_ARPA_INET_H)
check_include_file(fcntl.h HAVE_FCNTL_H)
check_include_file(inttypes.h HAVE_INTTYPES_H)
check_include_file(limits.h HAVE_LIMITS_H)
check_include_file(netdb.h HAVE_NETDB_H)
check_include_file(netinet/in.h HAVE_NETINET_IN_H)
check_include_file(pwd.h HAVE_PWD_H)
check_include_file(sys/socket.h HAVE_SYS_SOCKET_H)
check_include_file(sys/time.h HAVE_SYS_TIME_H)
check_include_file(syslog.h HAVE_SYSLOG_H)
check_include_file(time.h HAVE_TIME_H)
check_include_file(unistd.h HAVE_UNISTD_H)
include(CheckTypeSize)
# Checks for typedefs, structures, and compiler characteristics.
# AC_TYPE_SIZE_T
check_type_size("ssize_t" SIZEOF_SSIZE_T)
if(NOT SIZEOF_SSIZE_T)
# ssize_t is a signed type in POSIX storing at least -1.
# Set it to "int" to match the behavior of AC_TYPE_SSIZE_T (autotools).
set(ssize_t int)
endif()
include(CheckStructHasMember)
check_struct_has_member("struct tm" tm_gmtoff time.h HAVE_STRUCT_TM_TM_GMTOFF)
# Checks for library functions.
include(CheckFunctionExists)
check_function_exists(_Exit HAVE__EXIT)
check_function_exists(accept4 HAVE_ACCEPT4)
check_function_exists(mkostemp HAVE_MKOSTEMP)
include(CheckSymbolExists)
# XXX does this correctly detect initgroups (un)availability on cygwin?
check_symbol_exists(initgroups grp.h HAVE_DECL_INITGROUPS)
if(NOT HAVE_DECL_INITGROUPS AND HAVE_UNISTD_H)
# FreeBSD declares initgroups() in unistd.h
check_symbol_exists(initgroups unistd.h HAVE_DECL_INITGROUPS2)
if(HAVE_DECL_INITGROUPS2)
set(HAVE_DECL_INITGROUPS 1)
endif()
endif()
set(WARNCFLAGS)
if(CMAKE_C_COMPILER_ID MATCHES "MSVC")
if(ENABLE_WERROR)
set(WARNCFLAGS /WX)
endif()
else()
if(ENABLE_WERROR)
extract_valid_c_flags(WARNCFLAGS -Werror)
endif()
# For C compiler
# Please keep this list in sync with meson.build
extract_valid_c_flags(WARNCFLAGS
-Wall
-Wextra
-Wmissing-prototypes
-Wstrict-prototypes
-Wmissing-declarations
-Wpointer-arith
-Wdeclaration-after-statement
-Wformat-security
-Wwrite-strings
-Wshadow
-Winline
-Wnested-externs
-Wfloat-equal
-Wundef
-Wendif-labels
-Wempty-body
-Wcast-align
-Wclobbered
-Wvla
-Wpragmas
-Wunreachable-code
-Waddress
-Wattributes
-Wdiv-by-zero
-Wshorten-64-to-32
-Wconversion
-Wextended-offsetof
-Wformat-nonliteral
-Wlanguage-extension-token
-Wmissing-field-initializers
-Wmissing-noreturn
-Wmissing-variable-declarations
# -Wpadded # Not used because we cannot change public structs
-Wsign-conversion
# -Wswitch-enum # Not used because this basically disallows default case
-Wunreachable-code-break
-Wunused-macros
-Wunused-parameter
-Wredundant-decls
-Wheader-guard
-Wno-format-nonliteral # This is required because we pass format string as "const char*.
)
endif()
if(ENABLE_DEBUG)
set(DEBUGBUILD 1)
endif()
#add_definitions(-DHAVE_CONFIG_H)
#configure_file(cmakeconfig.h.in config.h)
# autotools-compatible names
# Sphinx expects relative paths in the .rst files. Use the fact that the files
# below are all one directory level deep.
file(RELATIVE_PATH top_srcdir ${CMAKE_CURRENT_BINARY_DIR}/dir ${CMAKE_CURRENT_SOURCE_DIR})
file(RELATIVE_PATH top_builddir ${CMAKE_CURRENT_BINARY_DIR}/dir ${CMAKE_CURRENT_BINARY_DIR})
set(abs_top_srcdir ${CMAKE_CURRENT_SOURCE_DIR})
set(abs_top_builddir ${CMAKE_CURRENT_BINARY_DIR})
# bzip2.pc (pkg-config file)
set(prefix ${CMAKE_INSTALL_PREFIX})
set(exec_prefix ${CMAKE_INSTALL_PREFIX})
set(bindir ${CMAKE_INSTALL_FULL_BINDIR})
set(sbindir ${CMAKE_INSTALL_FULL_SBINDIR})
set(libdir ${CMAKE_INSTALL_FULL_LIBDIR})
set(includedir ${CMAKE_INSTALL_FULL_INCLUDEDIR})
set(VERSION ${PACKAGE_VERSION})
configure_file(
bzip2.pc.in
${CMAKE_CURRENT_BINARY_DIR}/bzip2.pc
@ONLY)
install(FILES ${CMAKE_CURRENT_BINARY_DIR}/bzip2.pc
DESTINATION ${CMAKE_INSTALL_LIBDIR}/pkgconfig)
#
# The build targets.
# In a larger project, the following would be in subdirectories and
# These targets would be included with `add_subdirectory()`
#
set(BZ2_SOURCES
blocksort.c
huffman.c
crctable.c
randtable.c
compress.c
decompress.c
bzlib.c)
# The bz2 OBJECT-library, required for bzip2, bzip2recover.
add_library(bz2_ObjLib OBJECT)
target_sources(bz2_ObjLib
PRIVATE ${BZ2_SOURCES}
PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/bzlib_private.h
INTERFACE ${CMAKE_CURRENT_SOURCE_DIR}/bzlib.h)
# Windows resource file
set(BZ2_RES "")
if(WIN32)
configure_file(
version.rc.in
${CMAKE_CURRENT_BINARY_DIR}/version.rc
@ONLY)
set(BZ2_RES ${CMAKE_CURRENT_BINARY_DIR}/version.rc)
endif()
if(ENABLE_SHARED_LIB)
# The libbz2 shared library.
add_library(bz2 SHARED ${BZ2_RES})
target_sources(bz2
PRIVATE ${BZ2_SOURCES}
${CMAKE_CURRENT_SOURCE_DIR}/libbz2.def
PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/bzlib_private.h
INTERFACE ${CMAKE_CURRENT_SOURCE_DIR}/bzlib.h)
# Always use '-fPIC'/'-fPIE' option for shared libraries.
set_property(TARGET bz2 PROPERTY POSITION_INDEPENDENT_CODE ON)
set_target_properties(bz2 PROPERTIES
COMPILE_FLAGS "${WARNCFLAGS}"
VERSION ${LT_VERSION} SOVERSION ${LT_SOVERSION})
install(TARGETS bz2 DESTINATION ${CMAKE_INSTALL_LIBDIR})
install(FILES bzlib.h DESTINATION ${CMAKE_INSTALL_INCLUDEDIR})
if(USE_OLD_SONAME)
# Hack to support the old libbz2.so.1.0 version by including an extra copy.
# Technically the old SONAME is not libtool compatible.
# This hack is to support binary compatibility with libbz2 in some distro packages.
if(UNIX AND NOT APPLE)
add_library(bz2_old_soname SHARED ${BZ2_RES})
target_sources(bz2_old_soname
PRIVATE ${BZ2_SOURCES}
${CMAKE_CURRENT_SOURCE_DIR}/libbz2.def
PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/bzlib_private.h
INTERFACE ${CMAKE_CURRENT_SOURCE_DIR}/bzlib.h
)
set_target_properties(bz2_old_soname PROPERTIES
COMPILE_FLAGS "${WARNCFLAGS}"
VERSION ${LT_SOVERSION}.${LT_AGE} SOVERSION ${LT_SOVERSION}.${LT_AGE}
OUTPUT_NAME bz2
)
install(TARGETS bz2_old_soname DESTINATION ${CMAKE_INSTALL_LIBDIR})
endif()
endif()
endif()
if(ENABLE_STATIC_LIB)
# The libbz2 static library.
add_library(bz2_static STATIC)
target_sources(bz2_static
PRIVATE ${BZ2_SOURCES}
PUBLIC ${CMAKE_CURRENT_SOURCE_DIR}/bzlib_private.h
INTERFACE ${CMAKE_CURRENT_SOURCE_DIR}/bzlib.h)
# Use '-fPIC'/'-fPIE' option for static libraries by default.
# You may build with ENABLE_STATIC_LIB_IS_PIC=OFF to disable PIC for the static library.
if(ENABLE_STATIC_LIB_IS_PIC)
set_property(TARGET bz2_static PROPERTY POSITION_INDEPENDENT_CODE ON)
endif()
set_target_properties(bz2_static PROPERTIES
COMPILE_FLAGS "${WARNCFLAGS}"
VERSION ${LT_VERSION}
SOVERSION ${LT_SOVERSION}
ARCHIVE_OUTPUT_NAME bz2_static)
target_compile_definitions(bz2_static PUBLIC BZ2_STATICLIB)
install(TARGETS bz2_static DESTINATION ${CMAKE_INSTALL_LIBDIR})
install(FILES bzlib.h DESTINATION ${CMAKE_INSTALL_INCLUDEDIR})
endif()
if(ENABLE_APP)
# The bzip2 executable.
add_executable(bzip2)
target_sources(bzip2
PRIVATE bzip2.c)
target_link_libraries(bzip2
PRIVATE bz2_ObjLib)
if(WIN32)
target_compile_definitions(bzip2 PUBLIC BZ_LCCWIN32 BZ_UNIX=0)
else()
target_compile_definitions(bzip2 PUBLIC BZ_LCCWIN32=0 BZ_UNIX)
endif()
install(TARGETS bzip2 DESTINATION ${CMAKE_INSTALL_BINDIR})
# Create bzip2 copies bzcat and bunzip.
# The default behavior is altered in bzip2.c code by checking the program name.
install_target_symlink(bzip2 bzcat)
install_target_symlink(bzip2 bunzip)
# The bzip2recover executable.
add_executable(bzip2recover)
target_sources(bzip2recover
PRIVATE bzip2recover.c)
target_link_libraries(bzip2recover
PRIVATE bz2_ObjLib)
if(WIN32)
target_compile_definitions(bzip2recover PUBLIC BZ_LCCWIN32 BZ_UNIX=0)
else()
target_compile_definitions(bzip2recover PUBLIC BZ_LCCWIN32=0 BZ_UNIX)
endif()
install(TARGETS bzip2recover DESTINATION ${CMAKE_INSTALL_BINDIR})
if(ENABLE_EXAMPLES)
if(ENABLE_SHARED_LIB)
# The dlltest executable.
add_executable(dlltest)
target_sources(dlltest
PRIVATE dlltest.c)
target_link_libraries(dlltest bz2)
install(TARGETS dlltest DESTINATION ${CMAKE_INSTALL_BINDIR})
endif()
endif()
if(NOT WIN32)
# Install shell scripts, and renamed copies.
install(PROGRAMS bzdiff bzgrep bzmore
DESTINATION ${CMAKE_INSTALL_BINDIR})
install_script_symlink(bzdiff bzcmp)
install_script_symlink(bzgrep bzegrep)
install_script_symlink(bzgrep bzfgrep)
install_script_symlink(bzmore bzless)
endif()
endif()
if(ENABLE_APP AND Python3_FOUND)
enable_testing()
add_custom_target(check COMMAND ${CMAKE_CTEST_COMMAND})
add_subdirectory(tests)
endif()
add_subdirectory(man)
set(DOCGEN_EXECS xsltproc perl xmllint grep pdfxmltex pdftops)
if(ENABLE_DOCS)
foreach(EXEC IN LISTS DOCGEN_EXECS)
find_program(${EXEC}_EXEC ${EXEC})
if(NOT ${EXEC}_EXEC)
message(WARNING "Missing '${EXEC}', required to generate docs!")
set(MISSING_GENERATOR TRUE)
endif()
endforeach()
if(MISSING_GENERATOR)
message(FATAL_ERROR "Unable to generate docs.")
endif()
add_subdirectory(docs)
endif()
# The Summary Info.
string(TOUPPER "${CMAKE_BUILD_TYPE}" _build_type)
message(STATUS "Summary of build options:
Package version: ${VERSION}
Library version: ${LT_CURRENT}:${LT_REVISION}:${LT_AGE}
Install prefix: ${CMAKE_INSTALL_PREFIX}
Target system: ${CMAKE_SYSTEM_NAME}
Compiler:
Build type: ${CMAKE_BUILD_TYPE}
C compiler: ${CMAKE_C_COMPILER}
CFLAGS: ${CMAKE_C_FLAGS_${_build_type}} ${CMAKE_C_FLAGS}
WARNCFLAGS: ${WARNCFLAGS}
Test:
Python: ${Python3_FOUND} (${Python3_VERSION}, ${Python3_EXECUTABLE})
Docs:
Build docs: ${ENABLE_DOCS}
Features:
Applications: ${ENABLE_APP}
Examples: ${ENABLE_EXAMPLES}
")
if(ENABLE_LIB_ONLY_DISABLED_OTHERS)
message("Only the library will be built. To build other components "
"(such as applications and examples), set ENABLE_LIB_ONLY=OFF.")
endif()

27
thirdparty/bzip2/CMakeOptions.txt vendored
View file

@ -1,27 +0,0 @@
# Features that can be enabled for cmake (see CMakeLists.txt)
option(ENABLE_WERROR "Turn on compile time warnings")
option(ENABLE_DEBUG "Turn on debug output")
option(ENABLE_APP "Build applications (bzip2, and bzip2recover)"
${ENABLE_APP_DEFAULT})
option(ENABLE_TESTS "Build/enable unit tests."
${ENABLE_TESTS_DEFAULT})
option(ENABLE_DOCS "Generate documentation"
${ENABLE_DOCS_DEFAULT})
option(ENABLE_EXAMPLES "Build examples"
${ENABLE_EXAMPLES_DEFAULT})
option(ENABLE_LIB_ONLY "Build libbz2 only. This is a short hand for -DENABLE_APP=0 -DENABLE_EXAMPLES=0")
option(ENABLE_STATIC_LIB "Build libbz2 in static mode also")
option(ENABLE_SHARED_LIB "Build libbz2 as a shared library" ON)
option(USE_OLD_SONAME "Use libbz2.so.1.0 for compatibility with old Makefiles" OFF)
option(ENABLE_STATIC_LIB_IS_PIC "Enable position independent code for the static library" ON)

281
thirdparty/bzip2/COMPILING.md vendored
View file

@ -1,281 +0,0 @@
# Compiling bzip2
The following build systems are available for Bzip2:
* [Meson]: This is our preferred build system for Unix-like systems.
* [CMake]: Build tool for Unix and Windows.
Meson works for Unix-like OSes and Windows; nmake is only for Windows.
[Meson]: https://mesonbuild.com
[CMake]: https://cmake.org
> _Important note when compiling for Linux_:
>
> The SONAME for libbz2 for version 1.0 was: `libbz2.so.1.0`
> Some distros patched it to libbz2.so.1 to be supported by libtool.
> Others did not.
>
> We had to make a choice when switching from Makefiles -> CMake + Meson.
> So, the SONAME for libbz2 for version 1.1 is now: `libbz2.so.1`
>
> Distros that need it to be ABI compatible with the old SONAME may either:
> 1. Use CMake for the build with the option `-D USE_OLD_SONAME=ON`.
> This will build an extra copy of the library with the old SONAME.
>
> 2. Use `patchelf --set-soname` after the build to change the SONAME and
> install an extra symlink manually: `libbz2.so.1.0 -> libbz2.so.1.0.9`
>
> You can check the SONAME with: `objdump -p libbz2.so.1.0.9 | grep SONAME`
## Using Meson
Meson provides a [large number of built-in options](https://mesonbuild.com/Builtin-options.html)
to control compilation. A few important ones are listed below:
- -Ddefault_library=[static|shared|both], defaults to shared, if you wish to
statically link libbz2 into the binaries set this to `static`
- --backend : defaults to ninja, use `vs` if you want to use msbuild
- --unity : This enables a unity build (sometimes called a jumbo build), makes a single build faster but rebuilds slower
- -Dbuildtype=[debug|debugoptmized|release|minsize|plain] : Controls default optimization/debug generation args,
defaults to `debug`, use `plain` if you wish to pass your own cflags.
Meson recognizes environment variables like `$CFLAGS` and `$CC`.
It is recommended that you do not use `$CFLAGS`, and instead use `-Dc_args` and
`-DC_link_args`, as Meson will remember these even if you need to reconfigure
from scratch (such as when you update Meson), it will not remember `$CFLAGS`.
Meson will never change compilers once configured, so `$CC` is perfectly safe.
### Unix-like (Linux, *BSD, Cygwin, macOS)
You will need:
- Python 3.6 or newer (for 3.5 for Meson and 3.6 for the tests)
- Python's 'pytest' module, for running the tests.
- meson (Version 0.56 or newer)
- ninja
- pkg-config
- A C compiler such as GCC or Clang
Some linux distros package managers refer to ninja as ninja-build, fedora
and debian/ubuntu both do this. Your OS probably provides Meson, although
it may be too old, in that case you can use python3's pip to install Meson:
```sh
sudo pip3 install meson
```
or, for a user local install:
```sh
pip3 install --user meson
```
Once you have installed the dependencies, the following should work
to use the standard Meson configuration, a `builddir` for
compilation, and a `/usr` prefix for installation:
```sh
meson --prefix /usr builddir/
ninja -C builddir
meson test -C builddir --print-errorlogs
[sudo] ninja -C builddir install
```
You can use `meson configure builddir` to check configuration options.
Currently bzip only has one project specific option, which is to force the
generation of documentation on or off.
Ninja acepts many of the same arguments as make, although it will
automatically detect the number of CPU cores available and use an appropriate
number of threads.
### Windows
You will need:
- Python 3.6 or newer
- Meson
- Visual Studio 2015+ (Community edition is fine)
You can install Meson with Python's pip package manager:
```cmd
python -m pip install meson
```
or you can [download pre-bundled installers of meson directly from meson's github](https://github.com/mesonbuild/meson/releases).
Either should work fine for the purposes of building Bzip2.
You will also need pkg-config. There are many sources of pkg-config, I
recommend installing from Chocolatey because it's easy. Chocolatey can also
provide Ninja, though Ninja is not required on Windows if you want to use
msbuild.
If you want to use MSVC or a compatible compiler launch the associated
environment cmd to run Meson from; the environments required to make those
compilers work is quite complex otherwise.
Once you have all of that installed you can invoke Meson to configure the
build. By default Meson will generate a Ninja backend. If you would prefer to
use msbuild, pass the backend flag `--backend=vs`. MSVC (and compatible
compilers like clang-cl and ICL) work with Ninja as well.
```cmd
meson $builddir
ninja -C $builddir
meson test -C builddir --print-errorlogs
```
or:
```cmd
meson $builddir --backend=vs
cd $builddir
msbuild bzip2.sln /m
```
## Using CMake
### Requirements
For Linux/Unix:
- Python 3.6 or newer (for 3.5 for Meson and 3.6 for the tests)
- CMake (Version 3.12 or newer)
- A C compiler such as GCC or Clang
For Windows:
- Python 3.6 or newer
- CMake
- Visual Studio 2015+ (Community edition is fine)
### Build instructions for Unix & Windows (CMake)
Bzip2 can be compiled with the [CMake] build system.
You can use these commands to build Bzip2 in a certain `build` directory.
#### Basic Release build
Linux/Unix:
```sh
mkdir build && cd build
cmake .. -DCMAKE_BUILD_TYPE="Release"
cmake --build .
ctest -V
```
Windows:
```ps1
mkdir build && cd build
cmake ..
cmake --build . --config Release
ctest -C Release -V
```
#### Basic Debug build
Linux/Unix:
```sh
mkdir build && cd build
cmake .. -DCMAKE_BUILD_TYPE="Debug"
cmake --build .
ctest -V
```
Windows:
```ps1
mkdir build && cd build
cmake ..
cmake --build . --config Debug
ctest -C Release -V
```
#### Build and install to a specific install location (prefix)
Linux/Unix:
```sh
mkdir build && cd build
cmake .. -DCMAKE_BUILD_TYPE="Release" -DCMAKE_INSTALL_PREFIX=install
cmake --build .
ctest -V
cmake --build . --target install
```
Windows:
```ps1
mkdir build && cd build
cmake .. -DCMAKE_INSTALL_PREFIX=install
cmake --build . --config Release
ctest -C Release -V
cmake --build . --config Release --target install
```
#### Other CMake Options
`ENABLE_EXAMPLES`: Default: `OFF`
Enabling this option will also build the "dlltest" example executable. E.g.:
```sh
mkdir build && cd build
cmake .. -DENABLE_EXAMPLES=ON
cmake --build .
```
`ENABLE_DOCS`: Default: `OFF`
Enabling this option will generate extra documentation. E.g.:
```sh
mkdir build && cd build
cmake .. -DENABLE_DOCS=ON
cmake --build .
```
`ENABLE_APP`: Default: `ON`
Disabling this option will prevent building `bzip` or any of the other programs
that come with `bzip2`. It will also disable the tests. E.g.:
```sh
mkdir build && cd build
cmake .. -DENABLE_APP=OFF
cmake --build .
```
`ENABLE_LIB_ONLY`: Default: `OFF`
Enabling this option is similar to disabling `ENABLE_APP`. Only libbz2 will be
compiled. It will also disable the tests. E.g.:
```sh
mkdir build && cd build
cmake .. -DENABLE_LIB_ONLY=ON
cmake --build .
```
`ENABLE_STATIC_LIB`: Default: `OFF`
Enabling this option will build a static version of libbz2.
If `ENABLE_SHARED_LIB` is also enabled, the apps will link with the shared one.
E.g.:
```sh
mkdir build && cd build
cmake .. -DENABLE_STATIC_LIB=ON
cmake --build .
```
`ENABLE_SHARED_LIB`: Default: `ON`
Disabling this option will not build a shared version of libbz2. You must enable
`ENABLE_STATIC_LIB` if you disable `ENABLE_SHARED_LIB`. E.g.:
```sh
mkdir build && cd build
cmake .. -DENABLE_SHARED_LIB=OFF -DENABLE_STATIC_LIB=ON
cmake --build .
```
`USE_OLD_SONAME`: Default: `OFF`
Enabling this option will build an extra copy of the shared library that uses
the old SONAME. This option is made available for some linux distributions that
still distribute libbz2 with the old SONAME. E.g.:
```sh
mkdir build && cd build
cmake .. -DUSE_OLD_SONAME=ON
cmake --build .
```
`ENABLE_STATIC_LIB_IS_PIC`: Default: `ON`
Disabling this option will make it so that building the static library will not
use the '-fPIC'/'-fPIE' compiler options. You may need to disable this option if
your compiler cannot generate position-independent code for your platform. E.g.:
```sh
mkdir build && cd build
cmake .. -DENABLE_STATIC_LIB_IS_PIC=OFF
cmake --build .
```

42
thirdparty/bzip2/COPYING vendored
View file

@ -1,42 +0,0 @@
--------------------------------------------------------------------------
This program, "bzip2", the associated library "libbzip2", and all
documentation, are copyright (C) 1996-2010 Julian R Seward. All
rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. The origin of this software must not be misrepresented; you must
not claim that you wrote the original software. If you use this
software in a product, an acknowledgment in the product
documentation would be appreciated but is not required.
3. Altered source versions must be plainly marked as such, and must
not be misrepresented as being the original software.
4. The name of the author may not be used to endorse or promote
products derived from this software without specific prior written
permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS
OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Julian Seward, jseward@acm.org
bzip2/libbzip2 version 1.1.0 of 6 September 2010
--------------------------------------------------------------------------

468
thirdparty/bzip2/NEWS.md vendored
View file

@ -1,468 +0,0 @@
# Bzip2 News
## 1.1.0 (unreleased)
### General Announcements
[Micah Snyder](https://gitlab.com/micahsnyder) is the new maintainer of Bzip2
for feature development (v1.1+).
The Bzip2 feature development project is hosted on GitLab and can be found at
https://gitlab.com/bzip2/bzip2
Bzip2 version 1.0 is being maintained by Mark Wielaard at Sourceware and can be
found at https://sourceware.org/git/?p=bzip2.git
### Changes
Build system changes:
* Instead of the historical Makefile, Bzip2 now comes with two supported build
systems (Meson and CMake).
Either of the supported ones should build a shared library as per modern
practices. Please see the file [COMPILING.md](COMPILING.md) for details.
Important note when compiling for Linux:
* The SONAME for libbz2 for version 1.0 was: `libbz2.so.1.0`
Some distros patched it to `libbz2.so.1` to be supported by libtool.
Others did not.
We had to make a choice when switching from Makefiles -> CMake + Meson.
So, the SONAME for libbz2 for version 1.1 is now: `libbz2.so.1`
Distros that need it to be ABI compatible with the old SONAME may either:
1. Use CMake for the build with the option `-D USE_OLD_SONAME=ON`.
This will build an extra copy of the library with the old SONAME.
2. Use `patchelf --set-soname` after the build to change the SONAME and
install an extra symlink manually: `libbz2.so.1.0 -> libbz2.so.1.0.9`
You can check the SONAME with: `objdump -p libbz2.so.1.0.9 | grep SONAME`
Other changes, fixes:
* Use `O_CLOEXEC` for `bzopen()`. (Federico Mena Quintero)
* Fix `mingw` compilation. (Marty E. Plummer, Dylan Baker)
* Fix Visual Studio compilation. (Phil Ross)
* Don't let `bzip2recover` overwrite existing output files by default.
(Colin Phipps)
### Special Thanks
* Julian Seward for ceding maintainership and providing lots of advice
and interesting anecdotes.
* Mark Wielaard for constructing a repository based on tarballs from
the original releases. For a different construction by Evan Nemerson, see
https://gitlab.com/bzip2/bzip2/issues/7
* Federico Mena Quintero for his maintainership of the project June 2019 -
June 2021. Federico picked up the project from Julian Seward. He coordinated
and contributed to a patching and modernization effort, completed in this
release.
* Erich Córdoba and Jordan Petridis for the Continuous Integration
infrastructure.
* Phil Ross for the Windows build fixes.
* Dylan Baker for the Meson infrastructure.
* Micah Snyder for the CMake infrastructure.
* Stanislav Brabec for an Autotools infrastructure, which
became the basis for the Meson/CMake ones.
* Jens Korte for fixing up the README.
* All the people who submitted distribution-specific patches.
## 1.0.8 (13 Jul 19)
* Accept as many selectors as the file format allows.
This relaxes the fix for CVE-2019-12900 from 1.0.7
so that Bzip2 allows decompression of `.bz2` files that
use (too) many selectors again.
* Fix handling of large (> 4GB) files on Windows. (Phil Ross)
* Cleanup of `bzdiff` script so it doesn't use any bash extensions.
(Led)
* Cleanup of `bzgrep` script so it handle multiple archives
correctly. (Kristýna Streitová)
* There is now a bz2-files test suite at
https://sourceware.org/git/bzip2-tests.git
## 1.0.7 (27 Jun 19)
* Fix undefined behavior in the macros `SET_BH`, `CLEAR_BH`, & `ISSET_BH`
with the shift-left operator (Paul Kehrer).
* `bunzip2`: Fix return value when combining `--test` (`-t`) and `-q`.
* `bzip2recover`: Fix buffer overflow for large `argv[0]`. (Ivana Varekova)
https://bugzilla.redhat.com/show_bug.cgi?id=226979
* CVE-2016-3189 - Fix use-after-free in `bzip2recover`. (Jakub Martisko)
https://bugzilla.redhat.com/show_bug.cgi?id=1319648
* CVE-2019-12900 - Make sure `nSelectors` is not out of range.
Found through fuzzing karchive. (Albert Astals Cid)
## 1.0.6 (6 Sept 10)
* CVE-2010-0405: Security fix. This was reported by Mikolaj Izdebski.
* Make the documentation build on Ubuntu 10.04
## 1.0.5 (10 Dec 07)
Security fix only. Fixes CERT-FI 20469 as it applies to bzip2.
## 1.0.4 (20 Dec 06)
Fixes some minor bugs since the last version, 1.0.3.
* Fix file permissions race problem (CAN-2005-0953).
* Avoid possible segfault in BZ2_bzclose. From Coverity's NetBSD
scan.
* 'const'/prototype cleanups in the C code.
* Change default install location to /usr/local, and handle multiple
'make install's without error.
* Sanitise file names more carefully in bzgrep. Fixes CAN-2005-0758
to the extent that applies to bzgrep.
* Use 'mktemp' rather than 'tempfile' in bzdiff.
* Tighten up a couple of assertions in blocksort.c following automated
analysis.
* Fix minor doc/comment bugs.
## 1.0.3 (15 Feb 05)
Fixes some minor bugs since the last version, 1.0.2.
* Further robustification against corrupted compressed data.
There are currently no known bitstreams which can cause the
decompressor to crash, loop or access memory which does not
belong to it. If you are using bzip2 or the library to
decompress bitstreams from untrusted sources, an upgrade
to 1.0.3 is recommended. This fixes CAN-2005-1260.
* The documentation has been converted to XML, from which html
and pdf can be derived.
* Various minor bugs in the documentation have been fixed.
* Fixes for various compilation warnings with newer versions of
gcc, and on 64-bit platforms.
* The BZ_NO_STDIO cpp symbol was not properly observed in 1.0.2.
This has been fixed.
## 1.0.2
A bug fix release, addressing various minor issues which have appeared
in the 18 or so months since 1.0.1 was released. Most of the fixes
are to do with file-handling or documentation bugs. To the best of my
knowledge, there have been no data-loss-causing bugs reported in the
compression/decompression engine of 1.0.0 or 1.0.1.
Note that this release does not improve the rather crude build system
for Unix platforms. The general plan here is to autoconfiscate/
libtoolise 1.0.2 soon after release, and release the result as 1.1.0
or perhaps 1.2.0. That, however, is still just a plan at this point.
Here are the changes in 1.0.2. Bug-reporters and/or patch-senders in
parentheses.
* Fix an infinite segfault loop in 1.0.1 when a directory is
encountered in -f (force) mode.
(Trond Eivind Glomsrod, Nicholas Nethercote, Volker Schmidt)
* Avoid double fclose() of output file on certain I/O error paths.
(Solar Designer)
* Don't fail with internal error 1007 when fed a long stream (> 48MB)
of byte 251. Also print useful message suggesting that 1007s may be
caused by bad memory.
(noticed by Juan Pedro Vallejo, fixed by me)
* Fix uninitialised variable silly bug in demo prog dlltest.c.
(Jorj Bauer)
* Remove 512-MB limitation on recovered file size for bzip2recover
on selected platforms which support 64-bit ints. At the moment
all GCC supported platforms, and Win32.
(me, Alson van der Meulen)
* Hard-code header byte values, to give correct operation on platforms
using EBCDIC as their native character set (IBM's OS/390).
(Leland Lucius)
* Copy file access times correctly.
(Marty Leisner)
* Add distclean and check targets to Makefile.
(Michael Carmack)
* Parameterise use of ar and ranlib in Makefile. Also add $(LDFLAGS).
(Rich Ireland, Bo Thorsen)
* Pass -p (create parent dirs as needed) to mkdir during make install.
(Jeremy Fusco)
* Dereference symlinks when copying file permissions in -f mode.
(Volker Schmidt)
* Majorly simplify implementation of uInt64_qrm10.
(Bo Lindbergh)
* Check the input file still exists before deleting the output one,
when aborting in cleanUpAndFail().
(Joerg Prante, Robert Linden, Matthias Krings)
Also a bunch of patches courtesy of Philippe Troin, the Debian maintainer
of bzip2:
* Wrapper scripts (with manpages): bzdiff, bzgrep, bzmore.
* Spelling changes and minor enhancements in bzip2.1.
* Avoid race condition between creating the output file and setting its
interim permissions safely, by using fopen_output_safely().
No changes to bzip2recover since there is no issue with file
permissions there.
* do not print senseless report with -v when compressing an empty
file.
* bzcat -f works on non-bzip2 files.
* do not try to escape shell meta-characters on unix (the shell takes
care of these).
* added --fast and --best aliases for -1 -9 for gzip compatibility.
## 1.0.1
* Modified dlltest.c so it uses the new BZ2_ naming scheme.
* Modified makefile-msc to fix minor build probs on Win2k.
* Updated README.COMPILATION.PROBLEMS.
There are no functionality changes or bug fixes relative to version
1.0.0. This is just a documentation update + a fix for minor Win32
build problems. For almost everyone, upgrading from 1.0.0 to 1.0.1 is
utterly pointless. Don't bother.
## 1.0
Several minor bugfixes and enhancements:
* Large file support. The library uses 64-bit counters to
count the volume of data passing through it. bzip2.c
is now compiled with -D_FILE_OFFSET_BITS=64 to get large
file support from the C library. -v correctly prints out
file sizes greater than 4 gigabytes. All these changes have
been made without assuming a 64-bit platform or a C compiler
which supports 64-bit ints, so, except for the C library
aspect, they are fully portable.
* Decompression robustness. The library/program should be
robust to any corruption of compressed data, detecting and
handling _all_ corruption, instead of merely relying on
the CRCs. What this means is that the program should
never crash, given corrupted data, and the library should
always return BZ_DATA_ERROR.
* Fixed an obscure race-condition bug only ever observed on
Solaris, in which, if you were very unlucky and issued
control-C at exactly the wrong time, both input and output
files would be deleted.
* Don't run out of file handles on test/decompression when
large numbers of files have invalid magic numbers.
* Avoid library namespace pollution. Prefix all exported
symbols with BZ2_.
* Minor sorting enhancements from my DCC2000 paper.
* Advance the version number to 1.0, so as to counteract the
(false-in-this-case) impression some people have that programs
with version numbers less than 1.0 are in some way, experimental,
pre-release versions.
* Create an initial Makefile-libbz2_so to build a shared library.
Yes, I know I should really use libtool et al ...
* Make the program exit with 2 instead of 0 when decompression
fails due to a bad magic number (ie, an invalid bzip2 header).
Also exit with 1 (as the manual claims :-) whenever a diagnostic
message would have been printed AND the corresponding operation
is aborted, for example
bzip2: Output file xx already exists.
When a diagnostic message is printed but the operation is not
aborted, for example
bzip2: Can't guess original name for wurble -- using wurble.out
then the exit value 0 is returned, unless some other problem is
also detected.
I think it corresponds more closely to what the manual claims now.
## 0.9.5d
The only functional change is to make bzlibVersion() in the library
return the correct string. This has no effect whatsoever on the
functioning of the bzip2 program or library. Added a couple of casts
so the library compiles without warnings at level 3 in MS Visual
Studio 6.0. Included a Y2K statement in the file Y2K_INFO. All other
changes are minor documentation changes.
## 0.9.5c
Changed BZ_N_OVERSHOOT to be ... + 2 instead of ... + 1. The + 1
version could cause the sorted order to be wrong in some extremely
obscure cases. Also changed setting of quadrant in blocksort.c.
## 0.9.5b
Open stdin/stdout in binary mode for DJGPP.
## 0.9.5a
Major change: add a fallback sorting algorithm (blocksort.c)
to give reasonable behaviour even for very repetitive inputs.
Nuked --repetitive-best and --repetitive-fast since they are
no longer useful.
Minor changes: mostly a whole bunch of small changes/
bugfixes in the driver (bzip2.c). Changes pertaining to the
user interface are:
allow decompression of symlink'd files to stdout
decompress/test files even without .bz2 extension
give more accurate error messages for I/O errors
when compressing/decompressing to stdout, don't catch control-C
read flags from BZIP2 and BZIP environment variables
decline to break hard links to a file unless forced with -f
allow -c flag even with no filenames
preserve file ownerships as far as possible
make -s -1 give the expected block size (100k)
add a flag -q --quiet to suppress nonessential warnings
stop decoding flags after --, so files beginning in - can be handled
resolved inconsistent naming: bzcat or bz2cat ?
bzip2 --help now returns 0
Programming-level changes are:
fixed syntax error in GET_LL4 for Borland C++ 5.02
let bzBuffToBuffDecompress return BZ_DATA_ERROR{_MAGIC}
fix overshoot of mode-string end in bzopen_or_bzdopen
wrapped bzlib.h in #ifdef __cplusplus ... extern "C" { ... }
close file handles under all error conditions
added minor mods so it compiles with DJGPP out of the box
fixed Makefile so it doesn't give problems with BSD make
fix uninitialised memory reads in dlltest.c
Summary:
* Compression speed is much less sensitive to the input
data than in previous versions. Specifically, the very
slow performance caused by repetitive data is fixed.
* Many small improvements in file and flag handling.
* A Y2K statement.
## 0.9.0c
Fixed some problems in the library pertaining to some boundary cases.
This makes the library behave more correctly in those situations. The
fixes apply only to features (calls and parameters) not used by
bzip2.c, so the non-fixedness of them in previous versions has no
effect on reliability of bzip2.c.
In bzlib.c:
* made zero-length BZ_FLUSH work correctly in bzCompress().
* fixed bzWrite/bzRead to ignore zero-length requests.
* fixed bzread to correctly handle read requests after EOF.
* wrong parameter order in call to bzDecompressInit in
bzBuffToBuffDecompress. Fixed.
In compress.c:
* changed setting of nGroups in sendMTFValues() so as to
do a bit better on small files. This _does_ effect
bzip2.c.
## 0.9.0b
Fixed a problem with error reporting in bzip2.c. This does not effect
the library in any way. Problem is: versions 0.9.0 and 0.9.0a (of the
program proper) compress and decompress correctly, but give misleading
error messages (internal panics) when an I/O error occurs, instead of
reporting the problem correctly. This shouldn't give any data loss
(as far as I can see), but is confusing.
Made the inline declarations disappear for non-GCC compilers.
## 0.9.0a
Removed 'ranlib' from Makefile, since most modern Unix-es
don't need it, or even know about it.
## 0.9.0
First version after 0.1pl2.
* Approx 10% faster compression, 30% faster decompression
* -t (test mode) is a lot quicker
* Can decompress concatenated compressed files
* Programming interface, so programs can directly read/write .bz2 files
* Less restrictive (BSD-style) licensing
* Flag handling more compatible with GNU gzip
* Much more documentation, i.e., a proper user manual
* Hopefully, improved portability (at least of the library)
------------------------------------------------------------------
This file is part of bzip2/libbzip2, a program and library for
lossless, block-sorting data compression.
bzip2/libbzip2 version 1.1.0 of 6 September 2010
Copyright (C) 1996-2010 Julian Seward <jseward@acm.org>
Please read the WARNING, DISCLAIMER and PATENTS sections in the
README file.
This program is released under the terms of the license contained
in the file LICENSE.
------------------------------------------------------------------

45
thirdparty/bzip2/README.XML.STUFF vendored
View file

@ -1,45 +0,0 @@
----------------------------------------------------------------
This file is part of bzip2/libbzip2, a program and library for
lossless, block-sorting data compression.
bzip2/libbzip2 version 1.1.0 of 6 September 2010
Copyright (C) 1996-2010 Julian Seward <jseward@acm.org>
Please read the WARNING, DISCLAIMER and PATENTS sections in the
README file.
This program is released under the terms of the license contained
in the file LICENSE.
----------------------------------------------------------------
The script xmlproc.sh takes an xml file as input,
and processes it to create .pdf, .html or .ps output.
It uses format.pl, a perl script to format <pre> blocks nicely,
and add CDATA tags so writers do not have to use eg. &lt;
The file "entities.xml" must be edited to reflect current
version, year, etc.
Usage:
./xmlproc.sh -v manual.xml
Validates an xml file to ensure no dtd-compliance errors
./xmlproc.sh -html manual.xml
Output: manual.html
./xmlproc.sh -pdf manual.xml
Output: manual.pdf
./xmlproc.sh -ps manual.xml
Output: manual.ps
Notum bene:
- pdfxmltex barfs if given a filename with an underscore in it
- xmltex won't work yet - there's a bug in passivetex
which we are all waiting for Sebastian to fix.
So we are going the xml -> pdf -> ps route for the time being,
using pdfxmltex.

120
thirdparty/bzip2/README.md vendored
View file

@ -1,120 +0,0 @@
Bzip2
=====
This is Bzip2/libbz2; a program and library for lossless, block-sorting data
compression.
This document pertains to the Bzip2 feature development effort hosted on
[GitLab.com](https://gitlab.com/bzip2/bzip2).
The documentation here may differ from that on the Bzip2 1.0.x project page
maintained by Mark Wielaard on[sourceware.org](https://sourceware.org/bzip2/).
Copyright (C) 1996-2010 Julian Seward <jseward@acm.org>
Copyright (C) 2019-2020 Federico Mena Quintero <federico@gnome.org>
Copyright (C) 2021 [Micah Snyder](https://gitlab.com/micahsnyder).
Please read the [WARNING](#warning), [DISCLAIMER](#disclaimer) and
[PATENTS](#patents) sections in this file for important information.
This program is released under the terms of the license contained in the
[COPYING](COPYING) file.
------------------------------------------------------------------
This version is fully compatible with the previous public releases.
Complete documentation is available in Postscript form (manual.ps),
PDF (manual.pdf) or HTML (manual.html). A plain-text version of the
manual page is available as bzip2.txt.
## Community Code of Conduct
There is a code of conduct for contributors to Bzip2/libbz2.
Please see the [`code-of-conduct.md`](code-of-conduct.md) file.
## Contributing to Bzip2's development
The Bzip2 project is hosted on GitLab for feature development work.
It can be found at https://gitlab.com/bzip2/bzip2
Changes to be included in the next feature version are committed to the
`master` branch.
Feature releases are maintained in `release/*` branches.
Long-term feature and experimental development will occur in feature branches.
*Feature branches are unstable.* Feature branches may be rebased and force-
pushed on occasion to keep them up-to-date and to resolve merge conflicts.
The `rustify` branch is a feature branch that represents an effort to
gradually port Bzip2 to [Rust](https://www.rust-lang.org).
## Report a Bug
Please report bugs via [GitLab Issues](https://gitlab.com/bzip2/bzip2/issues).
Before you create a new issue, please verify that no one else has already
reported the same issue.
## Compiling Bzip2 and libbz2
Please see the [`COMPILING.md`](COMPILING.md) file for details.
This includes instructions for building using Meson or CMake.
## WARNING
This program and library (attempts to) compress data by performing several
non-trivial transformations on it. Unless you are 100% familiar with *all* the
algorithms contained herein, and with the consequences of modifying them, you
should NOT meddle with the compression or decompression machinery.
Incorrect changes can and very likely *will* lead to disastrous loss of data.
**Please contact the maintainers if you want to modify the algorithms.**
## DISCLAIMER
**I TAKE NO RESPONSIBILITY FOR ANY LOSS OF DATA ARISING FROM THE USE OF THIS
PROGRAM/LIBRARY, HOWSOEVER CAUSED.**
Every compression of a file implies an assumption that the compressed file can
be decompressed to reproduce the original. Great efforts in design, coding and
testing have been made to ensure that this program works correctly.
However, the complexity of the algorithms, and, in particular, the presence of
various special cases in the code which occur with very low but non-zero
probability make it impossible to rule out the possibility of bugs remaining in
the program.
DO NOT COMPRESS ANY DATA WITH THIS PROGRAM UNLESS YOU ARE PREPARED TO ACCEPT
THE POSSIBILITY, HOWEVER SMALL, THAT THE DATA WILL NOT BE RECOVERABLE.
That is not to say this program is inherently unreliable.
Indeed, I very much hope the opposite is true.
Bzip2/libbz2 has been carefully constructed and extensively tested.
## PATENTS
To the best of my knowledge, Bzip2/libbz2 does not use any patented algorithms.
However, I do not have the resources to carry out a patent search.
Therefore I cannot give any guarantee of the above statement.
## Maintainers
As of June 2021, [Micah Snyder](https://gitlab.com/micahsnyder) is the
maintainer of Bzip2/libbz2 for feature development work (I.e. versions 1.1+).
The Bzip2 feature development project is hosted on GitLab and can be found at
https://gitlab.com/bzip2/bzip2
Bzip2 version 1.0 is maintained by [Mark Wielaard](https://www.klomp.org/mark/)
at Sourceware and can be found at https://sourceware.org/git/?p=bzip2.git
### Special thanks
Thanks to Julian Seward, the original author of Bzip2/libbz2, for creating the
program and making it a very compelling alternative to previous compression
programs back in the early 2000's. Thanks to Julian also for letting Federico,
Mark, and Micah carry on with the maintainership of the program.

1094
thirdparty/bzip2/blocksort.c vendored
View file

File diff suppressed because it is too large Load diff

1
thirdparty/bzip2/bz_version.h.in vendored
View file

@ -1 +0,0 @@
#define BZ_VERSION "@BZ_VERSION@"

76
thirdparty/bzip2/bzdiff vendored
View file

@ -1,76 +0,0 @@
#!/bin/sh
# sh is buggy on RS/6000 AIX 3.2. Replace above line with #!/bin/ksh
# Bzcmp/diff wrapped for bzip2,
# adapted from zdiff by Philippe Troin <phil@fifi.org> for Debian GNU/Linux.
# Bzcmp and bzdiff are used to invoke the cmp or the diff pro-
# gram on compressed files. All options specified are passed
# directly to cmp or diff. If only 1 file is specified, then
# the files compared are file1 and an uncompressed file1.gz.
# If two files are specified, then they are uncompressed (if
# necessary) and fed to cmp or diff. The exit status from cmp
# or diff is preserved.
PATH="/usr/bin:/bin:$PATH"; export PATH
prog=`echo $0 | sed 's|.*/||'`
case "$prog" in
*cmp) comp=${CMP-cmp} ;;
*) comp=${DIFF-diff} ;;
esac
OPTIONS=
FILES=
for ARG
do
case "$ARG" in
-*) OPTIONS="$OPTIONS $ARG";;
*) if test -f "$ARG"; then
FILES="$FILES $ARG"
else
echo "${prog}: $ARG not found or not a regular file"
exit 1
fi ;;
esac
done
if test -z "$FILES"; then
echo "Usage: $prog [${comp}_options] file [file]"
exit 1
fi
set $FILES
if test $# -eq 1; then
FILE=`echo "$1" | sed 's/.bz2$//'`
bzip2 -cd "$FILE.bz2" | $comp $OPTIONS - "$FILE"
STAT="$?"
elif test $# -eq 2; then
case "$1" in
*.bz2)
case "$2" in
*.bz2)
F=`echo "$2" | sed 's|.*/||;s|.bz2$||'`
tmp=`mktemp "${TMPDIR:-/tmp}"/bzdiff.XXXXXXXXXX` || {
echo 'cannot create a temporary file' >&2
exit 1
}
bzip2 -cdfq "$2" > "$tmp"
bzip2 -cdfq "$1" | $comp $OPTIONS - "$tmp"
STAT="$?"
/bin/rm -f "$tmp";;
*) bzip2 -cdfq "$1" | $comp $OPTIONS - "$2"
STAT="$?";;
esac;;
*) case "$2" in
*.bz2)
bzip2 -cdfq "$2" | $comp $OPTIONS "$1" -
STAT="$?";;
*) $comp $OPTIONS "$1" "$2"
STAT="$?";;
esac;;
esac
else
echo "Usage: $prog [${comp}_options] file [file]"
exit 1
fi
exit "$STAT"

85
thirdparty/bzip2/bzgrep vendored
View file

@ -1,85 +0,0 @@
#!/bin/sh
# Bzgrep wrapped for bzip2,
# adapted from zgrep by Philippe Troin <phil@fifi.org> for Debian GNU/Linux.
## zgrep notice:
## zgrep -- a wrapper around a grep program that decompresses files as needed
## Adapted from a version sent by Charles Levert <charles@comm.polymtl.ca>
PATH="/usr/bin:$PATH"; export PATH
prog=`echo $0 | sed 's|.*/||'`
case "$prog" in
*egrep) grep=${EGREP-grep -E} ;;
*fgrep) grep=${FGREP-grep -F} ;;
*) grep=${GREP-grep} ;;
esac
pat=""
while test $# -ne 0; do
case "$1" in
-e | -f) opt="$opt $1"; shift; pat="$1"
if test "$grep" = grep; then # grep is buggy with -e on SVR4
grep="grep -E"
fi;;
-A | -B) opt="$opt $1 $2"; shift;;
-*) opt="$opt $1";;
*) if test -z "$pat"; then
pat="$1"
else
break;
fi;;
esac
shift
done
if test -z "$pat"; then
echo "grep through bzip2 files"
echo "usage: $prog [grep_options] pattern [files]"
exit 1
fi
list=0
silent=0
op=`echo "$opt" | sed -e 's/ //g' -e 's/-//g'`
case "$op" in
*l*) list=1
esac
case "$op" in
*h*) silent=1
esac
if test $# -eq 0; then
bzip2 -cdfq | $grep $opt "$pat"
exit $?
fi
res=0
for i do
if test -f "$i"; then :; else if test -f "$i.bz2"; then i="$i.bz2"; fi; fi
if test $list -eq 1; then
bzip2 -cdfq "$i" | $grep $opt "$pat" 2>&1 > /dev/null && echo $i
r=$?
elif test $# -eq 1 -o $silent -eq 1; then
bzip2 -cdfq "$i" | $grep $opt "$pat"
r=$?
else
j=$(echo "$i" | sed 's/\\/&&/g;s/|/\\&/g;s/&/\\&/g')
j=`printf "%s" "$j" | tr '\n' ' '`
# A trick adapted from
# https://groups.google.com/forum/#!original/comp.unix.shell/x1345iu10eg/Nn1n-1r1uU0J
# that has the same effect as the following bash code:
# bzip2 -cdfq "$i" | $grep $opt "$pat" | sed "s|^|${j}:|"
# r=${PIPESTATUS[1]}
exec 3>&1
eval `
exec 4>&1 >&3 3>&-
{
bzip2 -cdfq "$i" 4>&-
} | {
$grep $opt "$pat" 4>&-; echo "r=$?;" >&4
} | sed "s|^|${j}:|"
`
fi
test "$r" -ne 0 && res="$r"
done
exit $res

2029
thirdparty/bzip2/bzip2.c vendored
View file

File diff suppressed because it is too large Load diff

28
thirdparty/bzip2/bzip2.doap vendored
View file

@ -1,28 +0,0 @@
<Project xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#"
xmlns:foaf="http://xmlns.com/foaf/0.1/"
xmlns="http://usefulinc.com/ns/doap#">
<name xml:lang="en">Bzip2</name>
<shortdesc xml:lang="en">Program and library for lossless, block-sorting data compression</shortdesc>
<description>
Bzip2/libbz2 is a program and library for lossless, block-sorting
data compression. It is used widely to compress file archives and
as a component of compressed data streams.
</description>
<homepage rdf:resource="https://gitlab.com/bzip2/bzip2" />
<download-page rdf:resource="https://gitlab.com/bzip2/bzip2/-/tags" />
<bug-database rdf:resource="https://gitlab.com/bzip2/bzip2/issues" />
<programming-language>C</programming-language>
<programming-language>Rust</programming-language>
<maintainer>
<foaf:Person>
<foaf:name>Micah Snyder</foaf:name>
<foaf:mbox rdf:resource="mailto:micah.d.snyder@gmail.com" />
</foaf:Person>
</maintainer>
</Project>

11
thirdparty/bzip2/bzip2.pc.in vendored
View file

@ -1,11 +0,0 @@
prefix=@prefix@
exec_prefix=@exec_prefix@
bindir=@bindir@
libdir=@libdir@
includedir=@includedir@
Name: bzip2
Description: Lossless, block-sorting data compression
Version: @VERSION@
Libs: -L${libdir} -lbz2
Cflags: -I${includedir}

390
thirdparty/bzip2/bzip2.txt vendored
View file

@ -1,390 +0,0 @@
NAME
bzip2, bunzip2 - a block-sorting file compressor, v1.0.6
bzcat - decompresses files to stdout
bzip2recover - recovers data from damaged bzip2 files
SYNOPSIS
bzip2 [ -cdfkqstvzVL123456789 ] [ filenames ... ]
bunzip2 [ -fkvsVL ] [ filenames ... ]
bzcat [ -s ] [ filenames ... ]
bzip2recover filename
DESCRIPTION
bzip2 compresses files using the Burrows-Wheeler block
sorting text compression algorithm, and Huffman coding.
Compression is generally considerably better than that
achieved by more conventional LZ77/LZ78-based compressors,
and approaches the performance of the PPM family of sta-
tistical compressors.
The command-line options are deliberately very similar to
those of GNU gzip, but they are not identical.
bzip2 expects a list of file names to accompany the com-
mand-line flags. Each file is replaced by a compressed
version of itself, with the name "original_name.bz2".
Each compressed file has the same modification date, per-
missions, and, when possible, ownership as the correspond-
ing original, so that these properties can be correctly
restored at decompression time. File name handling is
naive in the sense that there is no mechanism for preserv-
ing original file names, permissions, ownerships or dates
in filesystems which lack these concepts, or have serious
file name length restrictions, such as MS-DOS.
bzip2 and bunzip2 will by default not overwrite existing
files. If you want this to happen, specify the -f flag.
If no file names are specified, bzip2 compresses from
standard input to standard output. In this case, bzip2
will decline to write compressed output to a terminal, as
this would be entirely incomprehensible and therefore
pointless.
bunzip2 (or bzip2 -d) decompresses all specified files.
Files which were not created by bzip2 will be detected and
ignored, and a warning issued. bzip2 attempts to guess
the filename for the decompressed file from that of the
compressed file as follows:
filename.bz2 becomes filename
filename.bz becomes filename
filename.tbz2 becomes filename.tar
filename.tbz becomes filename.tar
anyothername becomes anyothername.out
If the file does not end in one of the recognised endings,
.bz2, .bz, .tbz2 or .tbz, bzip2 complains that it cannot
guess the name of the original file, and uses the original
name with .out appended.
As with compression, supplying no filenames causes decom-
pression from standard input to standard output.
bunzip2 will correctly decompress a file which is the con-
catenation of two or more compressed files. The result is
the concatenation of the corresponding uncompressed files.
Integrity testing (-t) of concatenated compressed files is
also supported.
You can also compress or decompress files to the standard
output by giving the -c flag. Multiple files may be com-
pressed and decompressed like this. The resulting outputs
are fed sequentially to stdout. Compression of multiple
files in this manner generates a stream containing multi-
ple compressed file representations. Such a stream can be
decompressed correctly only by bzip2 version 0.9.0 or
later. Earlier versions of bzip2 will stop after decom-
pressing the first file in the stream.
bzcat (or bzip2 -dc) decompresses all specified files to
the standard output.
bzip2 will read arguments from the environment variables
BZIP2 and BZIP, in that order, and will process them
before any arguments read from the command line. This
gives a convenient way to supply default arguments.
Compression is always performed, even if the compressed
file is slightly larger than the original. Files of less
than about one hundred bytes tend to get larger, since the
compression mechanism has a constant overhead in the
region of 50 bytes. Random data (including the output of
most file compressors) is coded at about 8.05 bits per
byte, giving an expansion of around 0.5%.
As a self-check for your protection, bzip2 uses 32-bit
CRCs to make sure that the decompressed version of a file
is identical to the original. This guards against corrup-
tion of the compressed data, and against undetected bugs
in bzip2 (hopefully very unlikely). The chances of data
corruption going undetected is microscopic, about one
chance in four billion for each file processed. Be aware,
though, that the check occurs upon decompression, so it
can only tell you that something is wrong. It can't help
you recover the original uncompressed data. You can use
bzip2recover to try to recover data from damaged files.
Return values: 0 for a normal exit, 1 for environmental
problems (file not found, invalid flags, I/O errors, &c),
2 to indicate a corrupt compressed file, 3 for an internal
consistency error (eg, bug) which caused bzip2 to panic.
OPTIONS
-c --stdout
Compress or decompress to standard output.
-d --decompress
Force decompression. bzip2, bunzip2 and bzcat are
really the same program, and the decision about
what actions to take is done on the basis of which
name is used. This flag overrides that mechanism,
and forces bzip2 to decompress.
-z --compress
The complement to -d: forces compression,
regardless of the invocation name.
-t --test
Check integrity of the specified file(s), but don't
decompress them. This really performs a trial
decompression and throws away the result.
-f --force
Force overwrite of output files. Normally, bzip2
will not overwrite existing output files. Also
forces bzip2 to break hard links to files, which it
otherwise wouldn't do.
bzip2 normally declines to decompress files which
don't have the correct magic header bytes. If
forced (-f), however, it will pass such files
through unmodified. This is how GNU gzip behaves.
-k --keep
Keep (don't delete) input files during compression
or decompression.
-s --small
Reduce memory usage, for compression, decompression
and testing. Files are decompressed and tested
using a modified algorithm which only requires 2.5
bytes per block byte. This means any file can be
decompressed in 2300k of memory, albeit at about
half the normal speed.
During compression, -s selects a block size of
200k, which limits memory use to around the same
figure, at the expense of your compression ratio.
In short, if your machine is low on memory (8
megabytes or less), use -s for everything. See
MEMORY MANAGEMENT below.
-q --quiet
Suppress non-essential warning messages. Messages
pertaining to I/O errors and other critical events
will not be suppressed.
-v --verbose
Verbose mode -- show the compression ratio for each
file processed. Further -v's increase the ver-
bosity level, spewing out lots of information which
is primarily of interest for diagnostic purposes.
-L --license -V --version
Display the software version, license terms and
conditions.
-1 (or --fast) to -9 (or --best)
Set the block size to 100 k, 200 k .. 900 k when
compressing. Has no effect when decompressing.
See MEMORY MANAGEMENT below. The --fast and --best
aliases are primarily for GNU gzip compatibility.
In particular, --fast doesn't make things signifi-
cantly faster. And --best merely selects the
default behaviour.
-- Treats all subsequent arguments as file names, even
if they start with a dash. This is so you can han-
dle files with names beginning with a dash, for
example: bzip2 -- -myfilename.
--repetitive-fast --repetitive-best
These flags are redundant in versions 0.9.5 and
above. They provided some coarse control over the
behaviour of the sorting algorithm in earlier ver-
sions, which was sometimes useful. 0.9.5 and above
have an improved algorithm which renders these
flags irrelevant.
MEMORY MANAGEMENT
bzip2 compresses large files in blocks. The block size
affects both the compression ratio achieved, and the
amount of memory needed for compression and decompression.
The flags -1 through -9 specify the block size to be
100,000 bytes through 900,000 bytes (the default) respec-
tively. At decompression time, the block size used for
compression is read from the header of the compressed
file, and bunzip2 then allocates itself just enough memory
to decompress the file. Since block sizes are stored in
compressed files, it follows that the flags -1 to -9 are
irrelevant to and so ignored during decompression.
Compression and decompression requirements, in bytes, can
be estimated as:
Compression: 400k + ( 8 x block size )
Decompression: 100k + ( 4 x block size ), or
100k + ( 2.5 x block size )
Larger block sizes give rapidly diminishing marginal
returns. Most of the compression comes from the first two
or three hundred k of block size, a fact worth bearing in
mind when using bzip2 on small machines. It is also
important to appreciate that the decompression memory
requirement is set at compression time by the choice of
block size.
For files compressed with the default 900k block size,
bunzip2 will require about 3700 kbytes to decompress. To
support decompression of any file on a 4 megabyte machine,
bunzip2 has an option to decompress using approximately
half this amount of memory, about 2300 kbytes. Decompres-
sion speed is also halved, so you should use this option
only where necessary. The relevant flag is -s.
In general, try and use the largest block size memory con-
straints allow, since that maximises the compression
achieved. Compression and decompression speed are virtu-
ally unaffected by block size.
Another significant point applies to files which fit in a
single block -- that means most files you'd encounter
using a large block size. The amount of real memory
touched is proportional to the size of the file, since the
file is smaller than a block. For example, compressing a
file 20,000 bytes long with the flag -9 will cause the
compressor to allocate around 7600k of memory, but only
touch 400k + 20000 * 8 = 560 kbytes of it. Similarly, the
decompressor will allocate 3700k but only touch 100k +
20000 * 4 = 180 kbytes.
Here is a table which summarises the maximum memory usage
for different block sizes. Also recorded is the total
compressed size for 14 files of the Calgary Text Compres-
sion Corpus totalling 3,141,622 bytes. This column gives
some feel for how compression varies with block size.
These figures tend to understate the advantage of larger
block sizes for larger files, since the Corpus is domi-
nated by smaller files.
Compress Decompress Decompress Corpus
Flag usage usage -s usage Size
-1 1200k 500k 350k 914704
-2 2000k 900k 600k 877703
-3 2800k 1300k 850k 860338
-4 3600k 1700k 1100k 846899
-5 4400k 2100k 1350k 845160
-6 5200k 2500k 1600k 838626
-7 6100k 2900k 1850k 834096
-8 6800k 3300k 2100k 828642
-9 7600k 3700k 2350k 828642
RECOVERING DATA FROM DAMAGED FILES
bzip2 compresses files in blocks, usually 900kbytes long.
Each block is handled independently. If a media or trans-
mission error causes a multi-block .bz2 file to become
damaged, it may be possible to recover data from the
undamaged blocks in the file.
The compressed representation of each block is delimited
by a 48-bit pattern, which makes it possible to find the
block boundaries with reasonable certainty. Each block
also carries its own 32-bit CRC, so damaged blocks can be
distinguished from undamaged ones.
bzip2recover is a simple program whose purpose is to
search for blocks in .bz2 files, and write each block out
into its own .bz2 file. You can then use bzip2 -t to test
the integrity of the resulting files, and decompress those
which are undamaged.
bzip2recover takes a single argument, the name of the dam-
aged file, and writes a number of files
"rec00001file.bz2", "rec00002file.bz2", etc, containing
the extracted blocks. The output filenames are
designed so that the use of wildcards in subsequent pro-
cessing -- for example, "bzip2 -dc rec*file.bz2 > recov-
ered_data" -- processes the files in the correct order.
bzip2recover should be of most use dealing with large .bz2
files, as these will contain many blocks. It is clearly
futile to use it on damaged single-block files, since a
damaged block cannot be recovered. If you wish to min-
imise any potential data loss through media or transmis-
sion errors, you might consider compressing with a smaller
block size.
PERFORMANCE NOTES
The sorting phase of compression gathers together similar
strings in the file. Because of this, files containing
very long runs of repeated symbols, like "aabaabaabaab
..." (repeated several hundred times) may compress more
slowly than normal. Versions 0.9.5 and above fare much
better than previous versions in this respect. The ratio
between worst-case and average-case compression time is in
the region of 10:1. For previous versions, this figure
was more like 100:1. You can use the -vvvv option to mon-
itor progress in great detail, if you want.
Decompression speed is unaffected by these phenomena.
bzip2 usually allocates several megabytes of memory to
operate in, and then charges all over it in a fairly ran-
dom fashion. This means that performance, both for com-
pressing and decompressing, is largely determined by the
speed at which your machine can service cache misses.
Because of this, small changes to the code to reduce the
miss rate have been observed to give disproportionately
large performance improvements. I imagine bzip2 will per-
form best on machines with very large caches.
CAVEATS
I/O error messages are not as helpful as they could be.
bzip2 tries hard to detect I/O errors and exit cleanly,
but the details of what the problem is sometimes seem
rather misleading.
This manual page pertains to version 1.1.0 of bzip2. Com-
pressed data created by this version is entirely forwards
and backwards compatible with the previous public
releases, versions 0.1pl2, 0.9.0, 0.9.5, 1.0.0, 1.0.1,
1.0.2 and above, but with the following exception: 0.9.0
and above can correctly decompress multiple concatenated
compressed files. 0.1pl2 cannot do this; it will stop
after decompressing just the first file in the stream.
bzip2recover versions prior to 1.0.2 used 32-bit integers
to represent bit positions in compressed files, so they
could not handle compressed files more than 512 megabytes
long. Versions 1.0.2 and above use 64-bit ints on some
platforms which support them (GNU supported targets, and
Windows). To establish whether or not bzip2recover was
built with such a limitation, run it without arguments.
In any event you can build yourself an unlimited version
if you can recompile it with MaybeUInt64 set to be an
unsigned 64-bit integer.
AUTHOR
Julian Seward, jseward@acm.org
https://gitlab.com/bzip2/bzip2
The ideas embodied in bzip2 are due to (at least) the fol-
lowing people: Michael Burrows and David Wheeler (for the
block sorting transformation), David Wheeler (again, for
the Huffman coder), Peter Fenwick (for the structured cod-
ing model in the original bzip, and many refinements), and
Alistair Moffat, Radford Neal and Ian Witten (for the
arithmetic coder in the original bzip). I am much
indebted for their help, support and advice. See the man-
ual in the source distribution for pointers to sources of
documentation. Christian von Roques encouraged me to look
for faster sorting algorithms, so as to speed up compres-
sion. Bela Lubkin encouraged me to improve the worst-case
compression performance. Donna Robinson XMLised the docu-
mentation. The bz* scripts are derived from those of GNU
gzip. Many people sent patches, helped with portability
problems, lent machines, gave advice and were generally
helpful.

544
thirdparty/bzip2/bzip2recover.c vendored
View file

@ -1,544 +0,0 @@
/*-----------------------------------------------------------*/
/*--- Block recoverer program for bzip2 ---*/
/*--- bzip2recover.c ---*/
/*-----------------------------------------------------------*/
/* ------------------------------------------------------------------
This file is part of bzip2/libbzip2, a program and library for
lossless, block-sorting data compression.
bzip2/libbzip2 version 1.1.0 of 6 September 2010
Copyright (C) 1996-2010 Julian Seward <jseward@acm.org>
Please read the WARNING, DISCLAIMER and PATENTS sections in the
README file.
This program is released under the terms of the license contained
in the file LICENSE.
------------------------------------------------------------------ */
/* This program is a complete hack and should be rewritten properly.
It isn't very complicated. */
#if BZ_UNIX
# include <fcntl.h>
# include <sys/types.h>
# include <sys/stat.h>
# include <unistd.h>
#endif
#include <stdio.h>
#include <errno.h>
#include <stdlib.h>
#include <string.h>
/* This program records bit locations in the file to be recovered.
That means that if 64-bit ints are not supported, we will not
be able to recover .bz2 files over 512MB (2^32 bits) long.
On GNU supported platforms, we take advantage of the 64-bit
int support to circumvent this problem. Ditto MSVC.
This change occurred in version 1.0.2; all prior versions have
the 512MB limitation.
*/
#ifdef __GNUC__
typedef unsigned long long int MaybeUInt64;
# define MaybeUInt64_FMT "%Lu"
#else
#ifdef _MSC_VER
typedef unsigned __int64 MaybeUInt64;
# define MaybeUInt64_FMT "%I64u"
#else
typedef unsigned int MaybeUInt64;
# define MaybeUInt64_FMT "%u"
#endif
#endif
typedef unsigned int UInt32;
typedef int Int32;
typedef unsigned char UChar;
typedef char Char;
typedef unsigned char Bool;
#define True ((Bool)1)
#define False ((Bool)0)
#define BZ_MAX_FILENAME 2000
Char inFileName[BZ_MAX_FILENAME];
Char outFileName[BZ_MAX_FILENAME];
Char progName[BZ_MAX_FILENAME];
MaybeUInt64 bytesOut = 0;
MaybeUInt64 bytesIn = 0;
/*---------------------------------------------------*/
/*--- Header bytes ---*/
/*---------------------------------------------------*/
#define BZ_HDR_B 0x42 /* 'B' */
#define BZ_HDR_Z 0x5a /* 'Z' */
#define BZ_HDR_h 0x68 /* 'h' */
#define BZ_HDR_0 0x30 /* '0' */
/*---------------------------------------------------*/
/*--- I/O errors ---*/
/*---------------------------------------------------*/
/*---------------------------------------------*/
static void readError ( void )
{
fprintf ( stderr,
"%s: I/O error reading `%s', possible reason follows.\n",
progName, inFileName );
perror ( progName );
fprintf ( stderr, "%s: warning: output file(s) may be incomplete.\n",
progName );
exit ( 1 );
}
/*---------------------------------------------*/
static void writeError ( void )
{
fprintf ( stderr,
"%s: I/O error reading `%s', possible reason follows.\n",
progName, inFileName );
perror ( progName );
fprintf ( stderr, "%s: warning: output file(s) may be incomplete.\n",
progName );
exit ( 1 );
}
/*---------------------------------------------*/
static void mallocFail ( Int32 n )
{
fprintf ( stderr,
"%s: malloc failed on request for %d bytes.\n",
progName, n );
fprintf ( stderr, "%s: warning: output file(s) may be incomplete.\n",
progName );
exit ( 1 );
}
/*---------------------------------------------*/
static void tooManyBlocks ( Int32 max_handled_blocks )
{
fprintf ( stderr,
"%s: `%s' appears to contain more than %d blocks\n",
progName, inFileName, max_handled_blocks );
fprintf ( stderr,
"%s: and cannot be handled. To fix, increase\n",
progName );
fprintf ( stderr,
"%s: BZ_MAX_HANDLED_BLOCKS in bzip2recover.c, and recompile.\n",
progName );
exit ( 1 );
}
/*---------------------------------------------------*/
/*--- Bit stream I/O ---*/
/*---------------------------------------------------*/
typedef
struct {
FILE* handle;
Int32 buffer;
Int32 buffLive;
Char mode;
}
BitStream;
/*---------------------------------------------*/
static BitStream* bsOpenReadStream ( FILE* stream )
{
BitStream *bs = malloc ( sizeof(BitStream) );
if (bs == NULL) mallocFail ( sizeof(BitStream) );
bs->handle = stream;
bs->buffer = 0;
bs->buffLive = 0;
bs->mode = 'r';
return bs;
}
/*---------------------------------------------*/
static BitStream* bsOpenWriteStream ( FILE* stream )
{
BitStream *bs = malloc ( sizeof(BitStream) );
if (bs == NULL) mallocFail ( sizeof(BitStream) );
bs->handle = stream;
bs->buffer = 0;
bs->buffLive = 0;
bs->mode = 'w';
return bs;
}
/*---------------------------------------------*/
static void bsPutBit ( BitStream* bs, Int32 bit )
{
if (bs->buffLive == 8) {
Int32 retVal = putc ( (UChar) bs->buffer, bs->handle );
if (retVal == EOF) writeError();
bytesOut++;
bs->buffLive = 1;
bs->buffer = bit & 0x1;
} else {
bs->buffer = ( (bs->buffer << 1) | (bit & 0x1) );
bs->buffLive++;
};
}
/*---------------------------------------------*/
/*--
Returns 0 or 1, or 2 to indicate EOF.
--*/
static Int32 bsGetBit ( BitStream* bs )
{
if (bs->buffLive > 0) {
bs->buffLive --;
return ( ((bs->buffer) >> (bs->buffLive)) & 0x1 );
} else {
Int32 retVal = getc ( bs->handle );
if ( retVal == EOF ) {
if (errno != 0) readError();
return 2;
}
bs->buffLive = 7;
bs->buffer = retVal;
return ( ((bs->buffer) >> 7) & 0x1 );
}
}
/*---------------------------------------------*/
static void bsClose ( BitStream* bs )
{
Int32 retVal;
if ( bs->mode == 'w' ) {
while ( bs->buffLive < 8 ) {
bs->buffLive++;
bs->buffer <<= 1;
};
retVal = putc ( (UChar) (bs->buffer), bs->handle );
if (retVal == EOF) writeError();
bytesOut++;
retVal = fflush ( bs->handle );
if (retVal == EOF) writeError();
}
retVal = fclose ( bs->handle );
if (retVal == EOF) {
if (bs->mode == 'w') writeError(); else readError();
}
free ( bs );
}
/*---------------------------------------------*/
static void bsPutUChar ( BitStream* bs, UChar c )
{
Int32 i;
for (i = 7; i >= 0; i--)
bsPutBit ( bs, (((UInt32) c) >> i) & 0x1 );
}
/*---------------------------------------------*/
static void bsPutUInt32 ( BitStream* bs, UInt32 c )
{
Int32 i;
for (i = 31; i >= 0; i--)
bsPutBit ( bs, (c >> i) & 0x1 );
}
/*---------------------------------------------*/
static Bool endsInBz2 ( Char* name )
{
Int32 n = strlen ( name );
if (n <= 4) return False;
return
(name[n-4] == '.' &&
name[n-3] == 'b' &&
name[n-2] == 'z' &&
name[n-1] == '2');
}
/* Same as from bzip2.c
*
* Opens a file, but refuses to overwrite an existing one.
*/
static
FILE* fopen_output_safely ( Char* name, const char* mode )
{
# if BZ_UNIX
FILE* fp;
int fh;
fh = open(name, O_WRONLY|O_CREAT|O_EXCL, S_IWUSR|S_IRUSR);
if (fh == -1) return NULL;
fp = fdopen(fh, mode);
if (fp == NULL) close(fh);
return fp;
# else
return fopen(name, mode);
# endif
}
/*---------------------------------------------------*/
/*--- ---*/
/*---------------------------------------------------*/
/* This logic isn't really right when it comes to Cygwin. */
#ifdef _WIN32
# define BZ_SPLIT_SYM '\\' /* path splitter on Windows platform */
#else
# define BZ_SPLIT_SYM '/' /* path splitter on Unix platform */
#endif
#define BLOCK_HEADER_HI 0x00003141UL
#define BLOCK_HEADER_LO 0x59265359UL
#define BLOCK_ENDMARK_HI 0x00001772UL
#define BLOCK_ENDMARK_LO 0x45385090UL
/* Increase if necessary. However, a .bz2 file with > 50000 blocks
would have an uncompressed size of at least 40GB, so the chances
are low you'll need to up this.
*/
#define BZ_MAX_HANDLED_BLOCKS 50000
MaybeUInt64 bStart [BZ_MAX_HANDLED_BLOCKS];
MaybeUInt64 bEnd [BZ_MAX_HANDLED_BLOCKS];
MaybeUInt64 rbStart[BZ_MAX_HANDLED_BLOCKS];
MaybeUInt64 rbEnd [BZ_MAX_HANDLED_BLOCKS];
Int32 main ( Int32 argc, Char** argv )
{
FILE* inFile;
FILE* outFile;
BitStream* bsIn, *bsWr;
Int32 b, wrBlock, currBlock, rbCtr;
MaybeUInt64 bitsRead;
UInt32 buffHi, buffLo, blockCRC;
Char* p;
strncpy ( progName, argv[0], BZ_MAX_FILENAME-1);
progName[BZ_MAX_FILENAME-1]='\0';
inFileName[0] = outFileName[0] = 0;
fprintf ( stderr,
"bzip2recover 1.0.6: extracts blocks from damaged .bz2 files.\n" );
if (argc != 2) {
fprintf ( stderr, "%s: usage is `%s damaged_file_name'.\n",
progName, progName );
switch (sizeof(MaybeUInt64)) {
case 8:
fprintf(stderr,
"\trestrictions on size of recovered file: None\n");
break;
case 4:
fprintf(stderr,
"\trestrictions on size of recovered file: 512 MB\n");
fprintf(stderr,
"\tto circumvent, recompile with MaybeUInt64 as an\n"
"\tunsigned 64-bit int.\n");
break;
default:
fprintf(stderr,
"\tsizeof(MaybeUInt64) is not 4 or 8 -- "
"configuration error.\n");
break;
}
exit(1);
}
if (strlen(argv[1]) >= BZ_MAX_FILENAME-20) {
fprintf ( stderr,
"%s: supplied filename is suspiciously (>= %d chars) long. Bye!\n",
progName, (int)strlen(argv[1]) );
exit(1);
}
strcpy ( inFileName, argv[1] );
inFile = fopen ( inFileName, "rb" );
if (inFile == NULL) {
fprintf ( stderr, "%s: can't read `%s'\n", progName, inFileName );
exit(1);
}
bsIn = bsOpenReadStream ( inFile );
fprintf ( stderr, "%s: searching for block boundaries ...\n", progName );
bitsRead = 0;
buffHi = buffLo = 0;
currBlock = 0;
bStart[currBlock] = 0;
rbCtr = 0;
while (True) {
b = bsGetBit ( bsIn );
bitsRead++;
if (b == 2) {
if (bitsRead >= bStart[currBlock] &&
(bitsRead - bStart[currBlock]) >= 40) {
bEnd[currBlock] = bitsRead-1;
if (currBlock > 0)
fprintf ( stderr, " block %d runs from " MaybeUInt64_FMT
" to " MaybeUInt64_FMT " (incomplete)\n",
currBlock, bStart[currBlock], bEnd[currBlock] );
} else
currBlock--;
break;
}
buffHi = (buffHi << 1) | (buffLo >> 31);
buffLo = (buffLo << 1) | (b & 1);
if ( ( (buffHi & 0x0000ffff) == BLOCK_HEADER_HI
&& buffLo == BLOCK_HEADER_LO)
||
( (buffHi & 0x0000ffff) == BLOCK_ENDMARK_HI
&& buffLo == BLOCK_ENDMARK_LO)
) {
if (bitsRead > 49) {
bEnd[currBlock] = bitsRead-49;
} else {
bEnd[currBlock] = 0;
}
if (currBlock > 0 &&
(bEnd[currBlock] - bStart[currBlock]) >= 130) {
fprintf ( stderr, " block %d runs from " MaybeUInt64_FMT
" to " MaybeUInt64_FMT "\n",
rbCtr+1, bStart[currBlock], bEnd[currBlock] );
rbStart[rbCtr] = bStart[currBlock];
rbEnd[rbCtr] = bEnd[currBlock];
rbCtr++;
}
if (currBlock >= BZ_MAX_HANDLED_BLOCKS)
tooManyBlocks(BZ_MAX_HANDLED_BLOCKS);
currBlock++;
bStart[currBlock] = bitsRead;
}
}
bsClose ( bsIn );
/*-- identified blocks run from 1 to rbCtr inclusive. --*/
if (rbCtr < 1) {
fprintf ( stderr,
"%s: sorry, I couldn't find any block boundaries.\n",
progName );
exit(1);
};
fprintf ( stderr, "%s: splitting into blocks\n", progName );
inFile = fopen ( inFileName, "rb" );
if (inFile == NULL) {
fprintf ( stderr, "%s: can't open `%s'\n", progName, inFileName );
exit(1);
}
bsIn = bsOpenReadStream ( inFile );
/*-- placate gcc's dataflow analyser --*/
blockCRC = 0; bsWr = 0;
bitsRead = 0;
outFile = NULL;
wrBlock = 0;
while (True) {
b = bsGetBit(bsIn);
if (b == 2) break;
buffHi = (buffHi << 1) | (buffLo >> 31);
buffLo = (buffLo << 1) | (b & 1);
if (bitsRead == 47+rbStart[wrBlock])
blockCRC = (buffHi << 16) | (buffLo >> 16);
if (outFile != NULL && bitsRead >= rbStart[wrBlock]
&& bitsRead <= rbEnd[wrBlock]) {
bsPutBit ( bsWr, b );
}
bitsRead++;
if (bitsRead == rbEnd[wrBlock]+1) {
if (outFile != NULL) {
bsPutUChar ( bsWr, 0x17 ); bsPutUChar ( bsWr, 0x72 );
bsPutUChar ( bsWr, 0x45 ); bsPutUChar ( bsWr, 0x38 );
bsPutUChar ( bsWr, 0x50 ); bsPutUChar ( bsWr, 0x90 );
bsPutUInt32 ( bsWr, blockCRC );
bsClose ( bsWr );
outFile = NULL;
}
if (wrBlock >= rbCtr) break;
wrBlock++;
} else
if (bitsRead == rbStart[wrBlock]) {
/* Create the output file name, correctly handling leading paths.
(31.10.2001 by Sergey E. Kusikov) */
Char* split;
Int32 ofs, k;
for (k = 0; k < BZ_MAX_FILENAME; k++)
outFileName[k] = 0;
strcpy (outFileName, inFileName);
split = strrchr (outFileName, BZ_SPLIT_SYM);
if (split == NULL) {
split = outFileName;
} else {
++split;
}
/* Now split points to the start of the basename. */
ofs = split - outFileName;
sprintf (split, "rec%5d", wrBlock+1);
for (p = split; *p != 0; p++) if (*p == ' ') *p = '0';
strcat (outFileName, inFileName + ofs);
if ( !endsInBz2(outFileName)) strcat ( outFileName, ".bz2" );
fprintf ( stderr, " writing block %d to `%s' ...\n",
wrBlock+1, outFileName );
outFile = fopen_output_safely ( outFileName, "wb" );
if (outFile == NULL) {
fprintf ( stderr, "%s: can't write `%s'\n",
progName, outFileName );
exit(1);
}
bsWr = bsOpenWriteStream ( outFile );
bsPutUChar ( bsWr, BZ_HDR_B );
bsPutUChar ( bsWr, BZ_HDR_Z );
bsPutUChar ( bsWr, BZ_HDR_h );
bsPutUChar ( bsWr, BZ_HDR_0 + 9 );
bsPutUChar ( bsWr, 0x31 ); bsPutUChar ( bsWr, 0x41 );
bsPutUChar ( bsWr, 0x59 ); bsPutUChar ( bsWr, 0x26 );
bsPutUChar ( bsWr, 0x53 ); bsPutUChar ( bsWr, 0x59 );
}
}
fprintf ( stderr, "%s: finished\n", progName );
return 0;
}
/*-----------------------------------------------------------*/
/*--- end bzip2recover.c ---*/
/*-----------------------------------------------------------*/

1580
thirdparty/bzip2/bzlib.c vendored
View file

File diff suppressed because it is too large Load diff

287
thirdparty/bzip2/bzlib.h vendored
View file

@ -1,287 +0,0 @@
/*-------------------------------------------------------------*/
/*--- Public header file for the library. ---*/
/*--- bzlib.h ---*/
/*-------------------------------------------------------------*/
/* ------------------------------------------------------------------
This file is part of bzip2/libbzip2, a program and library for
lossless, block-sorting data compression.
bzip2/libbzip2 version 1.1.0 of 6 September 2010
Copyright (C) 1996-2010 Julian Seward <jseward@acm.org>
Please read the WARNING, DISCLAIMER and PATENTS sections in the
README file.
This program is released under the terms of the license contained
in the file LICENSE.
------------------------------------------------------------------ */
#ifndef _BZLIB_H
#define _BZLIB_H
#ifdef __cplusplus
extern "C" {
#endif
#define BZ_RUN 0
#define BZ_FLUSH 1
#define BZ_FINISH 2
#define BZ_OK 0
#define BZ_RUN_OK 1
#define BZ_FLUSH_OK 2
#define BZ_FINISH_OK 3
#define BZ_STREAM_END 4
#define BZ_SEQUENCE_ERROR (-1)
#define BZ_PARAM_ERROR (-2)
#define BZ_MEM_ERROR (-3)
#define BZ_DATA_ERROR (-4)
#define BZ_DATA_ERROR_MAGIC (-5)
#define BZ_IO_ERROR (-6)
#define BZ_UNEXPECTED_EOF (-7)
#define BZ_OUTBUFF_FULL (-8)
#define BZ_CONFIG_ERROR (-9)
typedef
struct {
char *next_in;
unsigned int avail_in;
unsigned int total_in_lo32;
unsigned int total_in_hi32;
char *next_out;
unsigned int avail_out;
unsigned int total_out_lo32;
unsigned int total_out_hi32;
void *state;
void *(*bzalloc)(void *,int,int);
void (*bzfree)(void *,void *);
void *opaque;
}
bz_stream;
#ifndef BZ_IMPORT
#define BZ_EXPORT
#endif
#ifndef BZ_NO_STDIO
/* Need a definitition for FILE */
#include <stdio.h>
#endif
#ifdef _WIN32
# include <windows.h>
# ifdef small
/* windows.h define small to char */
# undef small
# endif
# ifndef WINAPI
# define WINAPI
# endif
# ifdef BZ_EXPORT
# define BZ_API(func) WINAPI func
# define BZ_EXTERN extern
# else
/* import windows dll dynamically */
# define BZ_API(func) (WINAPI * func)
# define BZ_EXTERN
# endif
#else
# define BZ_API(func) func
#endif
#ifndef BZ_EXTERN
#define BZ_EXTERN extern
#endif
/*-- Core (low-level) library functions --*/
BZ_EXTERN int BZ_API(BZ2_bzCompressInit) (
bz_stream* strm,
int blockSize100k,
int verbosity,
int workFactor
);
BZ_EXTERN int BZ_API(BZ2_bzCompress) (
bz_stream* strm,
int action
);
BZ_EXTERN int BZ_API(BZ2_bzCompressEnd) (
bz_stream* strm
);
BZ_EXTERN int BZ_API(BZ2_bzDecompressInit) (
bz_stream *strm,
int verbosity,
int small
);
BZ_EXTERN int BZ_API(BZ2_bzDecompress) (
bz_stream* strm
);
BZ_EXTERN int BZ_API(BZ2_bzDecompressEnd) (
bz_stream *strm
);
/*-- High(er) level library functions --*/
#ifndef BZ_NO_STDIO
#define BZ_MAX_UNUSED 5000
typedef void BZFILE;
BZ_EXTERN BZFILE* BZ_API(BZ2_bzReadOpen) (
int* bzerror,
FILE* f,
int verbosity,
int small,
void* unused,
int nUnused
);
BZ_EXTERN void BZ_API(BZ2_bzReadClose) (
int* bzerror,
BZFILE* b
);
BZ_EXTERN void BZ_API(BZ2_bzReadGetUnused) (
int* bzerror,
BZFILE* b,
void** unused,
int* nUnused
);
BZ_EXTERN int BZ_API(BZ2_bzRead) (
int* bzerror,
BZFILE* b,
void* buf,
int len
);
BZ_EXTERN BZFILE* BZ_API(BZ2_bzWriteOpen) (
int* bzerror,
FILE* f,
int blockSize100k,
int verbosity,
int workFactor
);
BZ_EXTERN void BZ_API(BZ2_bzWrite) (
int* bzerror,
BZFILE* b,
void* buf,
int len
);
BZ_EXTERN void BZ_API(BZ2_bzWriteClose) (
int* bzerror,
BZFILE* b,
int abandon,
unsigned int* nbytes_in,
unsigned int* nbytes_out
);
BZ_EXTERN void BZ_API(BZ2_bzWriteClose64) (
int* bzerror,
BZFILE* b,
int abandon,
unsigned int* nbytes_in_lo32,
unsigned int* nbytes_in_hi32,
unsigned int* nbytes_out_lo32,
unsigned int* nbytes_out_hi32
);
#endif
/*-- Utility functions --*/
BZ_EXTERN int BZ_API(BZ2_bzBuffToBuffCompress) (
char* dest,
unsigned int* destLen,
char* source,
unsigned int sourceLen,
int blockSize100k,
int verbosity,
int workFactor
);
BZ_EXTERN int BZ_API(BZ2_bzBuffToBuffDecompress) (
char* dest,
unsigned int* destLen,
char* source,
unsigned int sourceLen,
int small,
int verbosity
);
/*--
Code contributed by Yoshioka Tsuneo (tsuneo@rr.iij4u.or.jp)
to support better zlib compatibility.
This code is not _officially_ part of libbzip2 (yet);
I haven't tested it, documented it, or considered the
threading-safeness of it.
If this code breaks, please contact both Yoshioka and me.
--*/
BZ_EXTERN const char * BZ_API(BZ2_bzlibVersion) (
void
);
#ifndef BZ_NO_STDIO
BZ_EXTERN BZFILE * BZ_API(BZ2_bzopen) (
const char *path,
const char *mode
);
BZ_EXTERN BZFILE * BZ_API(BZ2_bzdopen) (
int fd,
const char *mode
);
BZ_EXTERN int BZ_API(BZ2_bzread) (
BZFILE* b,
void* buf,
int len
);
BZ_EXTERN int BZ_API(BZ2_bzwrite) (
BZFILE* b,
void* buf,
int len
);
BZ_EXTERN int BZ_API(BZ2_bzflush) (
BZFILE* b
);
BZ_EXTERN void BZ_API(BZ2_bzclose) (
BZFILE* b
);
BZ_EXTERN const char * BZ_API(BZ2_bzerror) (
BZFILE *b,
int *errnum
);
#endif
#ifdef __cplusplus
}
#endif
#endif
/*-------------------------------------------------------------*/
/*--- end bzlib.h ---*/
/*-------------------------------------------------------------*/

507
thirdparty/bzip2/bzlib_private.h vendored
View file

@ -1,507 +0,0 @@
/*-------------------------------------------------------------*/
/*--- Private header file for the library. ---*/
/*--- bzlib_private.h ---*/
/*-------------------------------------------------------------*/
/* ------------------------------------------------------------------
This file is part of bzip2/libbzip2, a program and library for
lossless, block-sorting data compression.
bzip2/libbzip2 version 1.1.0 of 6 September 2010
Copyright (C) 1996-2010 Julian Seward <jseward@acm.org>
Please read the WARNING, DISCLAIMER and PATENTS sections in the
README file.
This program is released under the terms of the license contained
in the file LICENSE.
------------------------------------------------------------------ */
#ifndef _BZLIB_PRIVATE_H
#define _BZLIB_PRIVATE_H
#include <stdlib.h>
#ifndef BZ_NO_STDIO
#include <stdio.h>
#include <ctype.h>
#include <string.h>
#endif
#include "bzlib.h"
/*-- General stuff. --*/
typedef char Char;
typedef unsigned char Bool;
typedef unsigned char UChar;
typedef int Int32;
typedef unsigned int UInt32;
typedef short Int16;
typedef unsigned short UInt16;
#define True ((Bool)1)
#define False ((Bool)0)
#ifndef __GNUC__
#define __inline__ /* */
#endif
#ifndef BZ_NO_STDIO
extern void BZ2_bz__AssertH__fail ( int errcode );
#define AssertH(cond,errcode) \
{ if (!(cond)) BZ2_bz__AssertH__fail ( errcode ); }
#if BZ_DEBUG
#define AssertD(cond,msg) \
{ if (!(cond)) { \
fprintf ( stderr, \
"\n\nlibbzip2(debug build): internal error\n\t%s\n", msg );\
exit(1); \
}}
#else
#define AssertD(cond,msg) /* */
#endif
#define VPrintf0(zf) \
fprintf(stderr,zf)
#define VPrintf1(zf,za1) \
fprintf(stderr,zf,za1)
#define VPrintf2(zf,za1,za2) \
fprintf(stderr,zf,za1,za2)
#define VPrintf3(zf,za1,za2,za3) \
fprintf(stderr,zf,za1,za2,za3)
#define VPrintf4(zf,za1,za2,za3,za4) \
fprintf(stderr,zf,za1,za2,za3,za4)
#define VPrintf5(zf,za1,za2,za3,za4,za5) \
fprintf(stderr,zf,za1,za2,za3,za4,za5)
#else
extern void bz_internal_error ( int errcode );
#define AssertH(cond,errcode) \
{ if (!(cond)) bz_internal_error ( errcode ); }
#define AssertD(cond,msg) do { } while (0)
#define VPrintf0(zf) do { } while (0)
#define VPrintf1(zf,za1) do { } while (0)
#define VPrintf2(zf,za1,za2) do { } while (0)
#define VPrintf3(zf,za1,za2,za3) do { } while (0)
#define VPrintf4(zf,za1,za2,za3,za4) do { } while (0)
#define VPrintf5(zf,za1,za2,za3,za4,za5) do { } while (0)
#endif
#define BZALLOC(nnn) (strm->bzalloc)(strm->opaque,(nnn),1)
#define BZFREE(ppp) (strm->bzfree)(strm->opaque,(ppp))
/*-- Header bytes. --*/
#define BZ_HDR_B 0x42 /* 'B' */
#define BZ_HDR_Z 0x5a /* 'Z' */
#define BZ_HDR_h 0x68 /* 'h' */
#define BZ_HDR_0 0x30 /* '0' */
/*-- Constants for the back end. --*/
#define BZ_MAX_ALPHA_SIZE 258
#define BZ_MAX_CODE_LEN 23
#define BZ_RUNA 0
#define BZ_RUNB 1
#define BZ_N_GROUPS 6
#define BZ_G_SIZE 50
#define BZ_N_ITERS 4
#define BZ_MAX_SELECTORS (2 + (900000 / BZ_G_SIZE))
/*-- Stuff for randomising repetitive blocks. --*/
extern Int32 BZ2_rNums[512];
#define BZ_RAND_DECLS \
Int32 rNToGo; \
Int32 rTPos \
#define BZ_RAND_INIT_MASK \
s->rNToGo = 0; \
s->rTPos = 0 \
#define BZ_RAND_MASK ((s->rNToGo == 1) ? 1 : 0)
#define BZ_RAND_UPD_MASK \
if (s->rNToGo == 0) { \
s->rNToGo = BZ2_rNums[s->rTPos]; \
s->rTPos++; \
if (s->rTPos == 512) s->rTPos = 0; \
} \
s->rNToGo--;
/*-- Stuff for doing CRCs. --*/
extern UInt32 BZ2_crc32Table[256];
#define BZ_INITIALISE_CRC(crcVar) \
{ \
crcVar = 0xffffffffL; \
}
#define BZ_FINALISE_CRC(crcVar) \
{ \
crcVar = ~(crcVar); \
}
#define BZ_UPDATE_CRC(crcVar,cha) \
{ \
crcVar = (crcVar << 8) ^ \
BZ2_crc32Table[(crcVar >> 24) ^ \
((UChar)cha)]; \
}
/*-- States and modes for compression. --*/
#define BZ_M_IDLE 1
#define BZ_M_RUNNING 2
#define BZ_M_FLUSHING 3
#define BZ_M_FINISHING 4
#define BZ_S_OUTPUT 1
#define BZ_S_INPUT 2
#define BZ_N_RADIX 2
#define BZ_N_QSORT 12
#define BZ_N_SHELL 18
#define BZ_N_OVERSHOOT (BZ_N_RADIX + BZ_N_QSORT + BZ_N_SHELL + 2)
/*-- Structure holding all the compression-side stuff. --*/
typedef
struct {
/* pointer back to the struct bz_stream */
bz_stream* strm;
/* mode this stream is in, and whether inputting */
/* or outputting data */
Int32 mode;
Int32 state;
/* remembers avail_in when flush/finish requested */
UInt32 avail_in_expect;
/* for doing the block sorting */
UInt32* arr1;
UInt32* arr2;
UInt32* ftab;
Int32 origPtr;
/* aliases for arr1 and arr2 */
UInt32* ptr;
UChar* block;
UInt16* mtfv;
UChar* zbits;
/* for deciding when to use the fallback sorting algorithm */
Int32 workFactor;
/* run-length-encoding of the input */
UInt32 state_in_ch;
Int32 state_in_len;
BZ_RAND_DECLS;
/* input and output limits and current posns */
Int32 nblock;
Int32 nblockMAX;
Int32 numZ;
Int32 state_out_pos;
/* map of bytes used in block */
Int32 nInUse;
Bool inUse[256];
UChar unseqToSeq[256];
/* the buffer for bit stream creation */
UInt32 bsBuff;
Int32 bsLive;
/* block and combined CRCs */
UInt32 blockCRC;
UInt32 combinedCRC;
/* misc administratium */
Int32 verbosity;
Int32 blockNo;
Int32 blockSize100k;
/* stuff for coding the MTF values */
Int32 nMTF;
Int32 mtfFreq [BZ_MAX_ALPHA_SIZE];
UChar selector [BZ_MAX_SELECTORS];
UChar selectorMtf[BZ_MAX_SELECTORS];
UChar len [BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE];
Int32 code [BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE];
Int32 rfreq [BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE];
/* second dimension: only 3 needed; 4 makes index calculations faster */
UInt32 len_pack[BZ_MAX_ALPHA_SIZE][4];
}
EState;
/*-- externs for compression. --*/
extern void
BZ2_blockSort ( EState* );
extern void
BZ2_compressBlock ( EState*, Bool );
extern void
BZ2_bsInitWrite ( EState* );
extern void
BZ2_hbAssignCodes ( Int32*, UChar*, Int32, Int32, Int32 );
extern void
BZ2_hbMakeCodeLengths ( UChar*, Int32*, Int32, Int32 );
/*-- states for decompression. --*/
#define BZ_X_IDLE 1
#define BZ_X_OUTPUT 2
#define BZ_X_MAGIC_1 10
#define BZ_X_MAGIC_2 11
#define BZ_X_MAGIC_3 12
#define BZ_X_MAGIC_4 13
#define BZ_X_BLKHDR_1 14
#define BZ_X_BLKHDR_2 15
#define BZ_X_BLKHDR_3 16
#define BZ_X_BLKHDR_4 17
#define BZ_X_BLKHDR_5 18
#define BZ_X_BLKHDR_6 19
#define BZ_X_BCRC_1 20
#define BZ_X_BCRC_2 21
#define BZ_X_BCRC_3 22
#define BZ_X_BCRC_4 23
#define BZ_X_RANDBIT 24
#define BZ_X_ORIGPTR_1 25
#define BZ_X_ORIGPTR_2 26
#define BZ_X_ORIGPTR_3 27
#define BZ_X_MAPPING_1 28
#define BZ_X_MAPPING_2 29
#define BZ_X_SELECTOR_1 30
#define BZ_X_SELECTOR_2 31
#define BZ_X_SELECTOR_3 32
#define BZ_X_CODING_1 33
#define BZ_X_CODING_2 34
#define BZ_X_CODING_3 35
#define BZ_X_MTF_1 36
#define BZ_X_MTF_2 37
#define BZ_X_MTF_3 38
#define BZ_X_MTF_4 39
#define BZ_X_MTF_5 40
#define BZ_X_MTF_6 41
#define BZ_X_ENDHDR_2 42
#define BZ_X_ENDHDR_3 43
#define BZ_X_ENDHDR_4 44
#define BZ_X_ENDHDR_5 45
#define BZ_X_ENDHDR_6 46
#define BZ_X_CCRC_1 47
#define BZ_X_CCRC_2 48
#define BZ_X_CCRC_3 49
#define BZ_X_CCRC_4 50
/*-- Constants for the fast MTF decoder. --*/
#define MTFA_SIZE 4096
#define MTFL_SIZE 16
/*-- Structure holding all the decompression-side stuff. --*/
typedef
struct {
/* pointer back to the struct bz_stream */
bz_stream* strm;
/* state indicator for this stream */
Int32 state;
/* for doing the final run-length decoding */
UChar state_out_ch;
Int32 state_out_len;
Bool blockRandomised;
BZ_RAND_DECLS;
/* the buffer for bit stream reading */
UInt32 bsBuff;
Int32 bsLive;
/* misc administratium */
Int32 blockSize100k;
Bool smallDecompress;
Int32 currBlockNo;
Int32 verbosity;
/* for undoing the Burrows-Wheeler transform */
Int32 origPtr;
UInt32 tPos;
Int32 k0;
Int32 unzftab[256];
Int32 nblock_used;
Int32 cftab[257];
Int32 cftabCopy[257];
/* for undoing the Burrows-Wheeler transform (FAST) */
UInt32 *tt;
/* for undoing the Burrows-Wheeler transform (SMALL) */
UInt16 *ll16;
UChar *ll4;
/* stored and calculated CRCs */
UInt32 storedBlockCRC;
UInt32 storedCombinedCRC;
UInt32 calculatedBlockCRC;
UInt32 calculatedCombinedCRC;
/* map of bytes used in block */
Int32 nInUse;
Bool inUse[256];
Bool inUse16[16];
UChar seqToUnseq[256];
/* for decoding the MTF values */
UChar mtfa [MTFA_SIZE];
Int32 mtfbase[256 / MTFL_SIZE];
UChar selector [BZ_MAX_SELECTORS];
UChar selectorMtf[BZ_MAX_SELECTORS];
UChar len [BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE];
Int32 limit [BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE];
Int32 base [BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE];
Int32 perm [BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE];
Int32 minLens[BZ_N_GROUPS];
/* save area for scalars in the main decompress code */
Int32 save_i;
Int32 save_j;
Int32 save_t;
Int32 save_alphaSize;
Int32 save_nGroups;
Int32 save_nSelectors;
Int32 save_EOB;
Int32 save_groupNo;
Int32 save_groupPos;
Int32 save_nextSym;
Int32 save_nblockMAX;
Int32 save_nblock;
Int32 save_es;
Int32 save_N;
Int32 save_curr;
Int32 save_zt;
Int32 save_zn;
Int32 save_zvec;
Int32 save_zj;
Int32 save_gSel;
Int32 save_gMinlen;
Int32* save_gLimit;
Int32* save_gBase;
Int32* save_gPerm;
}
DState;
/*-- Macros for decompression. --*/
#define BZ_GET_FAST(cccc) \
/* c_tPos is unsigned, hence test < 0 is pointless. */ \
if (s->tPos >= (UInt32)100000 * (UInt32)s->blockSize100k) return True; \
s->tPos = s->tt[s->tPos]; \
cccc = (UChar)(s->tPos & 0xff); \
s->tPos >>= 8;
#define BZ_GET_FAST_C(cccc) \
/* c_tPos is unsigned, hence test < 0 is pointless. */ \
if (c_tPos >= (UInt32)100000 * (UInt32)ro_blockSize100k) return True; \
c_tPos = c_tt[c_tPos]; \
cccc = (UChar)(c_tPos & 0xff); \
c_tPos >>= 8;
#define SET_LL4(i,n) \
{ if (((i) & 0x1) == 0) \
s->ll4[(i) >> 1] = (s->ll4[(i) >> 1] & 0xf0) | (n); else \
s->ll4[(i) >> 1] = (s->ll4[(i) >> 1] & 0x0f) | ((n) << 4); \
}
#define GET_LL4(i) \
((((UInt32)(s->ll4[(i) >> 1])) >> (((i) << 2) & 0x4)) & 0xF)
#define SET_LL(i,n) \
{ s->ll16[i] = (UInt16)(n & 0x0000ffff); \
SET_LL4(i, n >> 16); \
}
#define GET_LL(i) \
(((UInt32)s->ll16[i]) | (GET_LL4(i) << 16))
#define BZ_GET_SMALL(cccc) \
/* c_tPos is unsigned, hence test < 0 is pointless. */ \
if (s->tPos >= (UInt32)100000 * (UInt32)s->blockSize100k) return True; \
cccc = BZ2_indexIntoF ( s->tPos, s->cftab ); \
s->tPos = GET_LL(s->tPos);
/*-- externs for decompression. --*/
extern Int32
BZ2_indexIntoF ( Int32, Int32* );
extern Int32
BZ2_decompress ( DState* );
extern void
BZ2_hbCreateDecodeTables ( Int32*, Int32*, Int32*, UChar*,
Int32, Int32, Int32 );
#endif
/*-- BZ_NO_STDIO seems to make NULL disappear on some platforms. --*/
#ifdef BZ_NO_STDIO
#ifndef NULL
#define NULL 0
#endif
#endif
/*-------------------------------------------------------------*/
/*--- end bzlib_private.h ---*/
/*-------------------------------------------------------------*/

61
thirdparty/bzip2/bzmore vendored
View file

@ -1,61 +0,0 @@
#!/bin/sh
# Bzmore wrapped for bzip2,
# adapted from zmore by Philippe Troin <phil@fifi.org> for Debian GNU/Linux.
PATH="/usr/bin:$PATH"; export PATH
prog=`echo $0 | sed 's|.*/||'`
case "$prog" in
*less) more=less ;;
*) more=more ;;
esac
if test "`echo -n a`" = "-n a"; then
# looks like a SysV system:
n1=''; n2='\c'
else
n1='-n'; n2=''
fi
oldtty=`stty -g 2>/dev/null`
if stty -cbreak 2>/dev/null; then
cb='cbreak'; ncb='-cbreak'
else
# 'stty min 1' resets eof to ^a on both SunOS and SysV!
cb='min 1 -icanon'; ncb='icanon eof ^d'
fi
if test $? -eq 0 -a -n "$oldtty"; then
trap 'stty $oldtty 2>/dev/null; exit' 0 2 3 5 10 13 15
else
trap 'stty $ncb echo 2>/dev/null; exit' 0 2 3 5 10 13 15
fi
if test $# = 0; then
if test -t 0; then
echo usage: $prog files...
else
bzip2 -cdfq | eval $more
fi
else
FIRST=1
for FILE
do
if test $FIRST -eq 0; then
echo $n1 "--More--(Next file: $FILE)$n2"
stty $cb -echo 2>/dev/null
ANS=`dd bs=1 count=1 2>/dev/null`
stty $ncb echo 2>/dev/null
echo " "
if test "$ANS" = 'e' -o "$ANS" = 'q'; then
exit
fi
fi
if test "$ANS" != 's'; then
echo "------> $FILE <------"
bzip2 -cdfq "$FILE" | eval $more
fi
if test -t; then
FIRST=0
fi
done
fi

18
thirdparty/bzip2/cmake/ExtractValidFlags.cmake vendored
View file

@ -1,18 +0,0 @@
# Convenience function that checks the availability of certain
# C or C++ compiler flags and returns valid ones as a string.
include(CheckCCompilerFlag)
include(CheckCXXCompilerFlag)
function(extract_valid_c_flags varname)
set(valid_flags)
foreach(flag IN LISTS ARGN)
string(REGEX REPLACE "[^a-zA-Z0-9_]+" "_" flag_var ${flag})
set(flag_var "C_FLAG_${flag_var}")
check_c_compiler_flag("${flag}" "${flag_var}")
if(${flag_var})
set(valid_flags "${valid_flags} ${flag}")
endif()
endforeach()
set(${varname} "${valid_flags}" PARENT_SCOPE)
endfunction()

40
thirdparty/bzip2/cmake/FindCUnit.cmake vendored
View file

@ -1,40 +0,0 @@
# - Try to find cunit
# Once done this will define
# CUNIT_FOUND - System has cunit
# CUNIT_INCLUDE_DIRS - The cunit include directories
# CUNIT_LIBRARIES - The libraries needed to use cunit
find_package(PkgConfig QUIET)
pkg_check_modules(PC_CUNIT QUIET cunit)
find_path(CUNIT_INCLUDE_DIR
NAMES CUnit/CUnit.h
HINTS ${PC_CUNIT_INCLUDE_DIRS}
)
find_library(CUNIT_LIBRARY
NAMES cunit
HINTS ${PC_CUNIT_LIBRARY_DIRS}
)
if(CUNIT_INCLUDE_DIR)
set(_version_regex "^#define[ \t]+CU_VERSION[ \t]+\"([^\"]+)\".*")
file(STRINGS "${CUNIT_INCLUDE_DIR}/CUnit/CUnit.h"
CUNIT_VERSION REGEX "${_version_regex}")
string(REGEX REPLACE "${_version_regex}" "\\1"
CUNIT_VERSION "${CUNIT_VERSION}")
unset(_version_regex)
endif()
include(FindPackageHandleStandardArgs)
# handle the QUIETLY and REQUIRED arguments and set CUNIT_FOUND to TRUE
# if all listed variables are TRUE and the requested version matches.
find_package_handle_standard_args(CUnit REQUIRED_VARS
CUNIT_LIBRARY CUNIT_INCLUDE_DIR
VERSION_VAR CUNIT_VERSION)
if(CUNIT_FOUND)
set(CUNIT_LIBRARIES ${CUNIT_LIBRARY})
set(CUNIT_INCLUDE_DIRS ${CUNIT_INCLUDE_DIR})
endif()
mark_as_advanced(CUNIT_INCLUDE_DIR CUNIT_LIBRARY)

36
thirdparty/bzip2/cmake/FindValgrind.cmake vendored
View file

@ -1,36 +0,0 @@
#
# Find the Valgrind program.
#
# If found, will set: Valgrind_FOUND, Valgrind_VERSION, and Valgrind_EXECUTABLE
#
# If you have a custom install location for Valgrind, you can provide a hint
# by settings -DValgrind_HOME=<directory containing valgrind>
#
find_program(Valgrind_EXECUTABLE valgrind
HINTS "${Valgrind_HOME}"
PATH_SUFFIXES "bin"
)
if(Valgrind_EXECUTABLE)
execute_process(COMMAND "${Valgrind_EXECUTABLE}" --version
OUTPUT_VARIABLE Valgrind_VERSION_OUTPUT
ERROR_VARIABLE Valgrind_VERSION_ERROR
RESULT_VARIABLE Valgrind_VERSION_RESULT
)
if(NOT ${Valgrind_VERSION_RESULT} EQUAL 0)
message(STATUS "Valgrind not found: Failed to determine version.")
unset(Valgrind_EXECUTABLE)
else()
string(REGEX
MATCH "[0-9]+\\.[0-9]+(\\.[0-9]+)?(-nightly)?"
Valgrind_VERSION "${Valgrind_VERSION_OUTPUT}"
)
set(Valgrind_VERSION "${Valgrind_VERSION}")
set(Valgrind_FOUND 1)
message(STATUS "Valgrind found: ${Valgrind_EXECUTABLE}, ${Valgrind_VERSION}")
endif()
mark_as_advanced(Valgrind_EXECUTABLE Valgrind_VERSION)
else()
message(STATUS "Valgrind not found.")
endif()

26
thirdparty/bzip2/cmake/SymLink.cmake vendored
View file

@ -1,26 +0,0 @@
# Install a symlink of script to the "bin" directory.
# Not intended for use on Windows.
function(install_script_symlink original symlink)
add_custom_command(OUTPUT ${symlink}
COMMAND ${CMAKE_COMMAND} -E create_symlink ${original} ${symlink}
DEPENDS ${original}
COMMENT "Generating symbolic link ${symlink} of ${original}")
add_custom_target(${symlink}_tgt ALL DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/${symlink})
install(PROGRAMS ${CMAKE_CURRENT_BINARY_DIR}/${symlink} DESTINATION ${CMAKE_INSTALL_BINDIR})
endfunction()
# Install a symlink of binary target to the "bin" directory.
# On Windows, it will be a copy instead of a symlink.
function(install_target_symlink original symlink)
if(WIN32)
set(op copy)
set(symlink "${symlink}.exe")
else()
set(op create_symlink)
endif()
add_custom_command(TARGET ${original} POST_BUILD
COMMAND ${CMAKE_COMMAND} -E ${op} $<TARGET_FILE_NAME:${original}> ${symlink}
WORKING_DIRECTORY $<TARGET_FILE_DIR:${original}>
COMMENT "Generating symbolic link (or copy) ${symlink} of ${original}")
install(PROGRAMS $<TARGET_FILE_DIR:${original}>/${symlink} DESTINATION ${CMAKE_INSTALL_BINDIR})
endfunction()

11
thirdparty/bzip2/cmake/Version.cmake vendored
View file

@ -1,11 +0,0 @@
# Converts a version such as 1.2.255 to 0x0102ff
function(HexVersion version_hex_var major minor patch)
math(EXPR version_dec "${major} * 256 * 256 + ${minor} * 256 + ${patch}")
set(version_hex "0x")
foreach(i RANGE 5 0 -1)
math(EXPR num "(${version_dec} >> (4 * ${i})) & 15")
string(SUBSTRING "0123456789abcdef" ${num} 1 num_hex)
set(version_hex "${version_hex}${num_hex}")
endforeach()
set(${version_hex_var} "${version_hex}" PARENT_SCOPE)
endfunction()

74
thirdparty/bzip2/code-of-conduct.md vendored
View file

@ -1,74 +0,0 @@
# Contributor Covenant Code of Conduct
## Our Pledge
In the interest of fostering an open and welcoming environment, we as
contributors and maintainers pledge to making participation in our project and
our community a harassment-free experience for everyone, regardless of age, body
size, disability, ethnicity, gender identity and expression, level of experience,
education, socio-economic status, nationality, personal appearance, race,
religion, or sexual identity and orientation.
## Our Standards
Examples of behavior that contributes to creating a positive environment
include:
* Using welcoming and inclusive language
* Being respectful of differing viewpoints and experiences
* Gracefully accepting constructive criticism
* Focusing on what is best for the community
* Showing empathy towards other community members
Examples of unacceptable behavior by participants include:
* The use of sexualized language or imagery and unwelcome sexual attention or
advances
* Trolling, insulting/derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or electronic
address, without explicit permission
* Other conduct which could reasonably be considered inappropriate in a
professional setting
## Our Responsibilities
Project maintainers are responsible for clarifying the standards of acceptable
behavior and are expected to take appropriate and fair corrective action in
response to any instances of unacceptable behavior.
Project maintainers have the right and responsibility to remove, edit, or
reject comments, commits, code, wiki edits, issues, and other contributions
that are not aligned to this Code of Conduct, or to ban temporarily or
permanently any contributor for other behaviors that they deem inappropriate,
threatening, offensive, or harmful.
## Scope
This Code of Conduct applies both within project spaces and in public spaces
when an individual is representing the project or its community. Examples of
representing a project or community include using an official project e-mail
address, posting via an official social media account, or acting as an appointed
representative at an online or offline event. Representation of a project may be
further defined and clarified by project maintainers.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported by contacting the project team at federico@gnome.org. All
complaints will be reviewed and investigated and will result in a response that
is deemed necessary and appropriate to the circumstances. The project team is
obligated to maintain confidentiality with regard to the reporter of an incident.
Further details of specific enforcement policies may be posted separately.
Project maintainers who do not follow or enforce the Code of Conduct in good
faith may face temporary or permanent repercussions as determined by other
members of the project's leadership.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html
[homepage]: https://www.contributor-covenant.org

671
thirdparty/bzip2/compress.c vendored
View file

@ -1,671 +0,0 @@
/*-------------------------------------------------------------*/
/*--- Compression machinery (not incl block sorting) ---*/
/*--- compress.c ---*/
/*-------------------------------------------------------------*/
/* ------------------------------------------------------------------
This file is part of bzip2/libbzip2, a program and library for
lossless, block-sorting data compression.
bzip2/libbzip2 version 1.1.0 of 6 September 2010
Copyright (C) 1996-2010 Julian Seward <jseward@acm.org>
Please read the WARNING, DISCLAIMER and PATENTS sections in the
README file.
This program is released under the terms of the license contained
in the file LICENSE.
------------------------------------------------------------------ */
/* CHANGES
0.9.0 -- original version.
0.9.0a/b -- no changes in this file.
0.9.0c -- changed setting of nGroups in sendMTFValues()
so as to do a bit better on small files
*/
#include "bzlib_private.h"
/*---------------------------------------------------*/
/*--- Bit stream I/O ---*/
/*---------------------------------------------------*/
/*---------------------------------------------------*/
void BZ2_bsInitWrite ( EState* s )
{
s->bsLive = 0;
s->bsBuff = 0;
}
/*---------------------------------------------------*/
static
void bsFinishWrite ( EState* s )
{
while (s->bsLive > 0) {
s->zbits[s->numZ] = (UChar)(s->bsBuff >> 24);
s->numZ++;
s->bsBuff <<= 8;
s->bsLive -= 8;
}
}
/*---------------------------------------------------*/
#define bsNEEDW(nz) \
{ \
while (s->bsLive >= 8) { \
s->zbits[s->numZ] \
= (UChar)(s->bsBuff >> 24); \
s->numZ++; \
s->bsBuff <<= 8; \
s->bsLive -= 8; \
} \
}
/*---------------------------------------------------*/
static
__inline__
void bsW ( EState* s, Int32 n, UInt32 v )
{
bsNEEDW ( n );
s->bsBuff |= (v << (32 - s->bsLive - n));
s->bsLive += n;
}
/*---------------------------------------------------*/
static
void bsPutUInt32 ( EState* s, UInt32 u )
{
bsW ( s, 8, (u >> 24) & 0xffL );
bsW ( s, 8, (u >> 16) & 0xffL );
bsW ( s, 8, (u >> 8) & 0xffL );
bsW ( s, 8, u & 0xffL );
}
/*---------------------------------------------------*/
static
void bsPutUChar ( EState* s, UChar c )
{
bsW( s, 8, (UInt32)c );
}
/*---------------------------------------------------*/
/*--- The back end proper ---*/
/*---------------------------------------------------*/
/*---------------------------------------------------*/
static
void makeMaps_e ( EState* s )
{
Int32 i;
s->nInUse = 0;
for (i = 0; i < 256; i++)
if (s->inUse[i]) {
s->unseqToSeq[i] = s->nInUse;
s->nInUse++;
}
}
/*---------------------------------------------------*/
static
void generateMTFValues ( EState* s )
{
UChar yy[256];
Int32 i, j;
Int32 zPend;
Int32 wr;
Int32 EOB;
/*
After sorting (eg, here),
s->arr1 [ 0 .. s->nblock-1 ] holds sorted order,
and
((UChar*)s->arr2) [ 0 .. s->nblock-1 ]
holds the original block data.
The first thing to do is generate the MTF values,
and put them in
((UInt16*)s->arr1) [ 0 .. s->nblock-1 ].
Because there are strictly fewer or equal MTF values
than block values, ptr values in this area are overwritten
with MTF values only when they are no longer needed.
The final compressed bitstream is generated into the
area starting at
(UChar*) (&((UChar*)s->arr2)[s->nblock])
These storage aliases are set up in bzCompressInit(),
except for the last one, which is arranged in
compressBlock().
*/
UInt32* ptr = s->ptr;
UChar* block = s->block;
UInt16* mtfv = s->mtfv;
makeMaps_e ( s );
EOB = s->nInUse+1;
for (i = 0; i <= EOB; i++) s->mtfFreq[i] = 0;
wr = 0;
zPend = 0;
for (i = 0; i < s->nInUse; i++) yy[i] = (UChar) i;
for (i = 0; i < s->nblock; i++) {
UChar ll_i;
AssertD ( wr <= i, "generateMTFValues(1)" );
j = ptr[i]-1; if (j < 0) j += s->nblock;
ll_i = s->unseqToSeq[block[j]];
AssertD ( ll_i < s->nInUse, "generateMTFValues(2a)" );
if (yy[0] == ll_i) {
zPend++;
} else {
if (zPend > 0) {
zPend--;
while (True) {
if (zPend & 1) {
mtfv[wr] = BZ_RUNB; wr++;
s->mtfFreq[BZ_RUNB]++;
} else {
mtfv[wr] = BZ_RUNA; wr++;
s->mtfFreq[BZ_RUNA]++;
}
if (zPend < 2) break;
zPend = (zPend - 2) / 2;
};
zPend = 0;
}
{
register UChar rtmp;
register UChar* ryy_j;
register UChar rll_i;
rtmp = yy[1];
yy[1] = yy[0];
ryy_j = &(yy[1]);
rll_i = ll_i;
while ( rll_i != rtmp ) {
register UChar rtmp2;
ryy_j++;
rtmp2 = rtmp;
rtmp = *ryy_j;
*ryy_j = rtmp2;
};
yy[0] = rtmp;
j = ryy_j - &(yy[0]);
mtfv[wr] = j+1; wr++; s->mtfFreq[j+1]++;
}
}
}
if (zPend > 0) {
zPend--;
while (True) {
if (zPend & 1) {
mtfv[wr] = BZ_RUNB; wr++;
s->mtfFreq[BZ_RUNB]++;
} else {
mtfv[wr] = BZ_RUNA; wr++;
s->mtfFreq[BZ_RUNA]++;
}
if (zPend < 2) break;
zPend = (zPend - 2) / 2;
};
zPend = 0;
}
mtfv[wr] = EOB; wr++; s->mtfFreq[EOB]++;
s->nMTF = wr;
}
/*---------------------------------------------------*/
#define BZ_LESSER_ICOST 0
#define BZ_GREATER_ICOST 15
static
void sendMTFValues ( EState* s )
{
Int32 v, t, i, j, gs, ge, totc, bt, bc, iter;
Int32 nSelectors, alphaSize, minLen, maxLen, selCtr;
Int32 nGroups, nBytes;
/*--
UChar len [BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE];
is a global since the decoder also needs it.
Int32 code[BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE];
Int32 rfreq[BZ_N_GROUPS][BZ_MAX_ALPHA_SIZE];
are also globals only used in this proc.
Made global to keep stack frame size small.
--*/
UInt16 cost[BZ_N_GROUPS];
Int32 fave[BZ_N_GROUPS];
UInt16* mtfv = s->mtfv;
if (s->verbosity >= 3)
VPrintf3( " %d in block, %d after MTF & 1-2 coding, "
"%d+2 syms in use\n",
s->nblock, s->nMTF, s->nInUse );
alphaSize = s->nInUse+2;
for (t = 0; t < BZ_N_GROUPS; t++)
for (v = 0; v < alphaSize; v++)
s->len[t][v] = BZ_GREATER_ICOST;
/*--- Decide how many coding tables to use ---*/
AssertH ( s->nMTF > 0, 3001 );
if (s->nMTF < 200) nGroups = 2; else
if (s->nMTF < 600) nGroups = 3; else
if (s->nMTF < 1200) nGroups = 4; else
if (s->nMTF < 2400) nGroups = 5; else
nGroups = 6;
/*--- Generate an initial set of coding tables ---*/
{
Int32 nPart, remF, tFreq, aFreq;
nPart = nGroups;
remF = s->nMTF;
gs = 0;
while (nPart > 0) {
tFreq = remF / nPart;
ge = gs-1;
aFreq = 0;
while (aFreq < tFreq && ge < alphaSize-1) {
ge++;
aFreq += s->mtfFreq[ge];
}
if (ge > gs
&& nPart != nGroups && nPart != 1
&& ((nGroups-nPart) % 2 == 1)) {
aFreq -= s->mtfFreq[ge];
ge--;
}
if (s->verbosity >= 3)
VPrintf5( " initial group %d, [%d .. %d], "
"has %d syms (%4.1f%%)\n",
nPart, gs, ge, aFreq,
(100.0 * (float)aFreq) / (float)(s->nMTF) );
for (v = 0; v < alphaSize; v++)
if (v >= gs && v <= ge)
s->len[nPart-1][v] = BZ_LESSER_ICOST; else
s->len[nPart-1][v] = BZ_GREATER_ICOST;
nPart--;
gs = ge+1;
remF -= aFreq;
}
}
/*---
Iterate up to BZ_N_ITERS times to improve the tables.
---*/
for (iter = 0; iter < BZ_N_ITERS; iter++) {
for (t = 0; t < nGroups; t++) fave[t] = 0;
for (t = 0; t < nGroups; t++)
for (v = 0; v < alphaSize; v++)
s->rfreq[t][v] = 0;
/*---
Set up an auxiliary length table which is used to fast-track
the common case (nGroups == 6).
---*/
if (nGroups == 6) {
for (v = 0; v < alphaSize; v++) {
s->len_pack[v][0] = (s->len[1][v] << 16) | s->len[0][v];
s->len_pack[v][1] = (s->len[3][v] << 16) | s->len[2][v];
s->len_pack[v][2] = (s->len[5][v] << 16) | s->len[4][v];
}
}
nSelectors = 0;
totc = 0;
gs = 0;
while (True) {
/*--- Set group start & end marks. --*/
if (gs >= s->nMTF) break;
ge = gs + BZ_G_SIZE - 1;
if (ge >= s->nMTF) ge = s->nMTF-1;
/*--
Calculate the cost of this group as coded
by each of the coding tables.
--*/
for (t = 0; t < nGroups; t++) cost[t] = 0;
if (nGroups == 6 && 50 == ge-gs+1) {
/*--- fast track the common case ---*/
register UInt32 cost01, cost23, cost45;
register UInt16 icv;
cost01 = cost23 = cost45 = 0;
# define BZ_ITER(nn) \
icv = mtfv[gs+(nn)]; \
cost01 += s->len_pack[icv][0]; \
cost23 += s->len_pack[icv][1]; \
cost45 += s->len_pack[icv][2]; \
BZ_ITER(0); BZ_ITER(1); BZ_ITER(2); BZ_ITER(3); BZ_ITER(4);
BZ_ITER(5); BZ_ITER(6); BZ_ITER(7); BZ_ITER(8); BZ_ITER(9);
BZ_ITER(10); BZ_ITER(11); BZ_ITER(12); BZ_ITER(13); BZ_ITER(14);
BZ_ITER(15); BZ_ITER(16); BZ_ITER(17); BZ_ITER(18); BZ_ITER(19);
BZ_ITER(20); BZ_ITER(21); BZ_ITER(22); BZ_ITER(23); BZ_ITER(24);
BZ_ITER(25); BZ_ITER(26); BZ_ITER(27); BZ_ITER(28); BZ_ITER(29);
BZ_ITER(30); BZ_ITER(31); BZ_ITER(32); BZ_ITER(33); BZ_ITER(34);
BZ_ITER(35); BZ_ITER(36); BZ_ITER(37); BZ_ITER(38); BZ_ITER(39);
BZ_ITER(40); BZ_ITER(41); BZ_ITER(42); BZ_ITER(43); BZ_ITER(44);
BZ_ITER(45); BZ_ITER(46); BZ_ITER(47); BZ_ITER(48); BZ_ITER(49);
# undef BZ_ITER
cost[0] = cost01 & 0xffff; cost[1] = cost01 >> 16;
cost[2] = cost23 & 0xffff; cost[3] = cost23 >> 16;
cost[4] = cost45 & 0xffff; cost[5] = cost45 >> 16;
} else {
/*--- slow version which correctly handles all situations ---*/
for (i = gs; i <= ge; i++) {
UInt16 icv = mtfv[i];
for (t = 0; t < nGroups; t++) cost[t] += s->len[t][icv];
}
}
/*--
Find the coding table which is best for this group,
and record its identity in the selector table.
--*/
bc = 999999999; bt = -1;
for (t = 0; t < nGroups; t++)
if (cost[t] < bc) { bc = cost[t]; bt = t; };
totc += bc;
fave[bt]++;
s->selector[nSelectors] = bt;
nSelectors++;
/*--
Increment the symbol frequencies for the selected table.
--*/
if (nGroups == 6 && 50 == ge-gs+1) {
/*--- fast track the common case ---*/
# define BZ_ITUR(nn) s->rfreq[bt][ mtfv[gs+(nn)] ]++
BZ_ITUR(0); BZ_ITUR(1); BZ_ITUR(2); BZ_ITUR(3); BZ_ITUR(4);
BZ_ITUR(5); BZ_ITUR(6); BZ_ITUR(7); BZ_ITUR(8); BZ_ITUR(9);
BZ_ITUR(10); BZ_ITUR(11); BZ_ITUR(12); BZ_ITUR(13); BZ_ITUR(14);
BZ_ITUR(15); BZ_ITUR(16); BZ_ITUR(17); BZ_ITUR(18); BZ_ITUR(19);
BZ_ITUR(20); BZ_ITUR(21); BZ_ITUR(22); BZ_ITUR(23); BZ_ITUR(24);
BZ_ITUR(25); BZ_ITUR(26); BZ_ITUR(27); BZ_ITUR(28); BZ_ITUR(29);
BZ_ITUR(30); BZ_ITUR(31); BZ_ITUR(32); BZ_ITUR(33); BZ_ITUR(34);
BZ_ITUR(35); BZ_ITUR(36); BZ_ITUR(37); BZ_ITUR(38); BZ_ITUR(39);
BZ_ITUR(40); BZ_ITUR(41); BZ_ITUR(42); BZ_ITUR(43); BZ_ITUR(44);
BZ_ITUR(45); BZ_ITUR(46); BZ_ITUR(47); BZ_ITUR(48); BZ_ITUR(49);
# undef BZ_ITUR
} else {
/*--- slow version which correctly handles all situations ---*/
for (i = gs; i <= ge; i++)
s->rfreq[bt][ mtfv[i] ]++;
}
gs = ge+1;
}
if (s->verbosity >= 3) {
VPrintf2 ( " pass %d: size is %d, grp uses are ",
iter+1, totc/8 );
for (t = 0; t < nGroups; t++)
VPrintf1 ( "%d ", fave[t] );
VPrintf0 ( "\n" );
}
/*--
Recompute the tables based on the accumulated frequencies.
--*/
/* maxLen was changed from 20 to 17 in bzip2-1.0.3. See
comment in huffman.c for details. */
for (t = 0; t < nGroups; t++)
BZ2_hbMakeCodeLengths ( &(s->len[t][0]), &(s->rfreq[t][0]),
alphaSize, 17 /*20*/ );
}
AssertH( nGroups < 8, 3002 );
AssertH( nSelectors < 32768 &&
nSelectors <= BZ_MAX_SELECTORS,
3003 );
/*--- Compute MTF values for the selectors. ---*/
{
UChar pos[BZ_N_GROUPS], ll_i, tmp2, tmp;
for (i = 0; i < nGroups; i++) pos[i] = i;
for (i = 0; i < nSelectors; i++) {
ll_i = s->selector[i];
j = 0;
tmp = pos[j];
while ( ll_i != tmp ) {
j++;
tmp2 = tmp;
tmp = pos[j];
pos[j] = tmp2;
};
pos[0] = tmp;
s->selectorMtf[i] = j;
}
};
/*--- Assign actual codes for the tables. --*/
for (t = 0; t < nGroups; t++) {
minLen = 32;
maxLen = 0;
for (i = 0; i < alphaSize; i++) {
if (s->len[t][i] > maxLen) maxLen = s->len[t][i];
if (s->len[t][i] < minLen) minLen = s->len[t][i];
}
AssertH ( !(maxLen > 17 /*20*/ ), 3004 );
AssertH ( !(minLen < 1), 3005 );
BZ2_hbAssignCodes ( &(s->code[t][0]), &(s->len[t][0]),
minLen, maxLen, alphaSize );
}
/*--- Transmit the mapping table. ---*/
{
Bool inUse16[16];
for (i = 0; i < 16; i++) {
inUse16[i] = False;
for (j = 0; j < 16; j++)
if (s->inUse[i * 16 + j]) inUse16[i] = True;
}
nBytes = s->numZ;
for (i = 0; i < 16; i++)
if (inUse16[i]) bsW(s,1,1); else bsW(s,1,0);
for (i = 0; i < 16; i++)
if (inUse16[i])
for (j = 0; j < 16; j++) {
if (s->inUse[i * 16 + j]) bsW(s,1,1); else bsW(s,1,0);
}
if (s->verbosity >= 3)
VPrintf1( " bytes: mapping %d, ", s->numZ-nBytes );
}
/*--- Now the selectors. ---*/
nBytes = s->numZ;
bsW ( s, 3, nGroups );
bsW ( s, 15, nSelectors );
for (i = 0; i < nSelectors; i++) {
for (j = 0; j < s->selectorMtf[i]; j++) bsW(s,1,1);
bsW(s,1,0);
}
if (s->verbosity >= 3)
VPrintf1( "selectors %d, ", s->numZ-nBytes );
/*--- Now the coding tables. ---*/
nBytes = s->numZ;
for (t = 0; t < nGroups; t++) {
Int32 curr = s->len[t][0];
bsW ( s, 5, curr );
for (i = 0; i < alphaSize; i++) {
while (curr < s->len[t][i]) { bsW(s,2,2); curr++; /* 10 */ };
while (curr > s->len[t][i]) { bsW(s,2,3); curr--; /* 11 */ };
bsW ( s, 1, 0 );
}
}
if (s->verbosity >= 3)
VPrintf1 ( "code lengths %d, ", s->numZ-nBytes );
/*--- And finally, the block data proper ---*/
nBytes = s->numZ;
selCtr = 0;
gs = 0;
while (True) {
if (gs >= s->nMTF) break;
ge = gs + BZ_G_SIZE - 1;
if (ge >= s->nMTF) ge = s->nMTF-1;
AssertH ( s->selector[selCtr] < nGroups, 3006 );
if (nGroups == 6 && 50 == ge-gs+1) {
/*--- fast track the common case ---*/
UInt16 mtfv_i;
UChar* s_len_sel_selCtr
= &(s->len[s->selector[selCtr]][0]);
Int32* s_code_sel_selCtr
= &(s->code[s->selector[selCtr]][0]);
# define BZ_ITAH(nn) \
mtfv_i = mtfv[gs+(nn)]; \
bsW ( s, \
s_len_sel_selCtr[mtfv_i], \
s_code_sel_selCtr[mtfv_i] )
BZ_ITAH(0); BZ_ITAH(1); BZ_ITAH(2); BZ_ITAH(3); BZ_ITAH(4);
BZ_ITAH(5); BZ_ITAH(6); BZ_ITAH(7); BZ_ITAH(8); BZ_ITAH(9);
BZ_ITAH(10); BZ_ITAH(11); BZ_ITAH(12); BZ_ITAH(13); BZ_ITAH(14);
BZ_ITAH(15); BZ_ITAH(16); BZ_ITAH(17); BZ_ITAH(18); BZ_ITAH(19);
BZ_ITAH(20); BZ_ITAH(21); BZ_ITAH(22); BZ_ITAH(23); BZ_ITAH(24);
BZ_ITAH(25); BZ_ITAH(26); BZ_ITAH(27); BZ_ITAH(28); BZ_ITAH(29);
BZ_ITAH(30); BZ_ITAH(31); BZ_ITAH(32); BZ_ITAH(33); BZ_ITAH(34);
BZ_ITAH(35); BZ_ITAH(36); BZ_ITAH(37); BZ_ITAH(38); BZ_ITAH(39);
BZ_ITAH(40); BZ_ITAH(41); BZ_ITAH(42); BZ_ITAH(43); BZ_ITAH(44);
BZ_ITAH(45); BZ_ITAH(46); BZ_ITAH(47); BZ_ITAH(48); BZ_ITAH(49);
# undef BZ_ITAH
} else {
/*--- slow version which correctly handles all situations ---*/
for (i = gs; i <= ge; i++) {
bsW ( s,
s->len [s->selector[selCtr]] [mtfv[i]],
s->code [s->selector[selCtr]] [mtfv[i]] );
}
}
gs = ge+1;
selCtr++;
}
AssertH( selCtr == nSelectors, 3007 );
if (s->verbosity >= 3)
VPrintf1( "codes %d\n", s->numZ-nBytes );
}
/*---------------------------------------------------*/
void BZ2_compressBlock ( EState* s, Bool is_last_block )
{
if (s->nblock > 0) {
BZ_FINALISE_CRC ( s->blockCRC );
s->combinedCRC = (s->combinedCRC << 1) | (s->combinedCRC >> 31);
s->combinedCRC ^= s->blockCRC;
if (s->blockNo > 1) s->numZ = 0;
if (s->verbosity >= 2)
VPrintf4( " block %d: crc = 0x%08x, "
"combined CRC = 0x%08x, size = %d\n",
s->blockNo, s->blockCRC, s->combinedCRC, s->nblock );
BZ2_blockSort ( s );
}
s->zbits = (UChar*) (&((UChar*)s->arr2)[s->nblock]);
/*-- If this is the first block, create the stream header. --*/
if (s->blockNo == 1) {
BZ2_bsInitWrite ( s );
bsPutUChar ( s, BZ_HDR_B );
bsPutUChar ( s, BZ_HDR_Z );
bsPutUChar ( s, BZ_HDR_h );
bsPutUChar ( s, (UChar)(BZ_HDR_0 + s->blockSize100k) );
}
if (s->nblock > 0) {
bsPutUChar ( s, 0x31 ); bsPutUChar ( s, 0x41 );
bsPutUChar ( s, 0x59 ); bsPutUChar ( s, 0x26 );
bsPutUChar ( s, 0x53 ); bsPutUChar ( s, 0x59 );
/*-- Now the block's CRC, so it is in a known place. --*/
bsPutUInt32 ( s, s->blockCRC );
/*--
Now a single bit indicating (non-)randomisation.
As of version 0.9.5, we use a better sorting algorithm
which makes randomisation unnecessary. So always set
the randomised bit to 'no'. Of course, the decoder
still needs to be able to handle randomised blocks
so as to maintain backwards compatibility with
older versions of bzip2.
--*/
bsW(s,1,0);
bsW ( s, 24, s->origPtr );
generateMTFValues ( s );
sendMTFValues ( s );
}
/*-- If this is the last block, add the stream trailer. --*/
if (is_last_block) {
bsPutUChar ( s, 0x17 ); bsPutUChar ( s, 0x72 );
bsPutUChar ( s, 0x45 ); bsPutUChar ( s, 0x38 );
bsPutUChar ( s, 0x50 ); bsPutUChar ( s, 0x90 );
bsPutUInt32 ( s, s->combinedCRC );
if (s->verbosity >= 2)
VPrintf1( " final combined CRC = 0x%08x\n ", s->combinedCRC );
bsFinishWrite ( s );
}
}
/*-------------------------------------------------------------*/
/*--- end compress.c ---*/
/*-------------------------------------------------------------*/

104
thirdparty/bzip2/crctable.c vendored
View file

@ -1,104 +0,0 @@
/*-------------------------------------------------------------*/
/*--- Table for doing CRCs ---*/
/*--- crctable.c ---*/
/*-------------------------------------------------------------*/
/* ------------------------------------------------------------------
This file is part of bzip2/libbzip2, a program and library for
lossless, block-sorting data compression.
bzip2/libbzip2 version 1.1.0 of 6 September 2010
Copyright (C) 1996-2010 Julian Seward <jseward@acm.org>
Please read the WARNING, DISCLAIMER and PATENTS sections in the
README file.
This program is released under the terms of the license contained
in the file LICENSE.
------------------------------------------------------------------ */
#include "bzlib_private.h"
/*--
I think this is an implementation of the AUTODIN-II,
Ethernet & FDDI 32-bit CRC standard. Vaguely derived
from code by Rob Warnock, in Section 51 of the
comp.compression FAQ.
--*/
UInt32 BZ2_crc32Table[256] = {
/*-- Ugly, innit? --*/
0x00000000L, 0x04c11db7L, 0x09823b6eL, 0x0d4326d9L,
0x130476dcL, 0x17c56b6bL, 0x1a864db2L, 0x1e475005L,
0x2608edb8L, 0x22c9f00fL, 0x2f8ad6d6L, 0x2b4bcb61L,
0x350c9b64L, 0x31cd86d3L, 0x3c8ea00aL, 0x384fbdbdL,
0x4c11db70L, 0x48d0c6c7L, 0x4593e01eL, 0x4152fda9L,
0x5f15adacL, 0x5bd4b01bL, 0x569796c2L, 0x52568b75L,
0x6a1936c8L, 0x6ed82b7fL, 0x639b0da6L, 0x675a1011L,
0x791d4014L, 0x7ddc5da3L, 0x709f7b7aL, 0x745e66cdL,
0x9823b6e0L, 0x9ce2ab57L, 0x91a18d8eL, 0x95609039L,
0x8b27c03cL, 0x8fe6dd8bL, 0x82a5fb52L, 0x8664e6e5L,
0xbe2b5b58L, 0xbaea46efL, 0xb7a96036L, 0xb3687d81L,
0xad2f2d84L, 0xa9ee3033L, 0xa4ad16eaL, 0xa06c0b5dL,
0xd4326d90L, 0xd0f37027L, 0xddb056feL, 0xd9714b49L,
0xc7361b4cL, 0xc3f706fbL, 0xceb42022L, 0xca753d95L,
0xf23a8028L, 0xf6fb9d9fL, 0xfbb8bb46L, 0xff79a6f1L,
0xe13ef6f4L, 0xe5ffeb43L, 0xe8bccd9aL, 0xec7dd02dL,
0x34867077L, 0x30476dc0L, 0x3d044b19L, 0x39c556aeL,
0x278206abL, 0x23431b1cL, 0x2e003dc5L, 0x2ac12072L,
0x128e9dcfL, 0x164f8078L, 0x1b0ca6a1L, 0x1fcdbb16L,
0x018aeb13L, 0x054bf6a4L, 0x0808d07dL, 0x0cc9cdcaL,
0x7897ab07L, 0x7c56b6b0L, 0x71159069L, 0x75d48ddeL,
0x6b93dddbL, 0x6f52c06cL, 0x6211e6b5L, 0x66d0fb02L,
0x5e9f46bfL, 0x5a5e5b08L, 0x571d7dd1L, 0x53dc6066L,
0x4d9b3063L, 0x495a2dd4L, 0x44190b0dL, 0x40d816baL,
0xaca5c697L, 0xa864db20L, 0xa527fdf9L, 0xa1e6e04eL,
0xbfa1b04bL, 0xbb60adfcL, 0xb6238b25L, 0xb2e29692L,
0x8aad2b2fL, 0x8e6c3698L, 0x832f1041L, 0x87ee0df6L,
0x99a95df3L, 0x9d684044L, 0x902b669dL, 0x94ea7b2aL,
0xe0b41de7L, 0xe4750050L, 0xe9362689L, 0xedf73b3eL,
0xf3b06b3bL, 0xf771768cL, 0xfa325055L, 0xfef34de2L,
0xc6bcf05fL, 0xc27dede8L, 0xcf3ecb31L, 0xcbffd686L,
0xd5b88683L, 0xd1799b34L, 0xdc3abdedL, 0xd8fba05aL,
0x690ce0eeL, 0x6dcdfd59L, 0x608edb80L, 0x644fc637L,
0x7a089632L, 0x7ec98b85L, 0x738aad5cL, 0x774bb0ebL,
0x4f040d56L, 0x4bc510e1L, 0x46863638L, 0x42472b8fL,
0x5c007b8aL, 0x58c1663dL, 0x558240e4L, 0x51435d53L,
0x251d3b9eL, 0x21dc2629L, 0x2c9f00f0L, 0x285e1d47L,
0x36194d42L, 0x32d850f5L, 0x3f9b762cL, 0x3b5a6b9bL,
0x0315d626L, 0x07d4cb91L, 0x0a97ed48L, 0x0e56f0ffL,
0x1011a0faL, 0x14d0bd4dL, 0x19939b94L, 0x1d528623L,
0xf12f560eL, 0xf5ee4bb9L, 0xf8ad6d60L, 0xfc6c70d7L,
0xe22b20d2L, 0xe6ea3d65L, 0xeba91bbcL, 0xef68060bL,
0xd727bbb6L, 0xd3e6a601L, 0xdea580d8L, 0xda649d6fL,
0xc423cd6aL, 0xc0e2d0ddL, 0xcda1f604L, 0xc960ebb3L,
0xbd3e8d7eL, 0xb9ff90c9L, 0xb4bcb610L, 0xb07daba7L,
0xae3afba2L, 0xaafbe615L, 0xa7b8c0ccL, 0xa379dd7bL,
0x9b3660c6L, 0x9ff77d71L, 0x92b45ba8L, 0x9675461fL,
0x8832161aL, 0x8cf30badL, 0x81b02d74L, 0x857130c3L,
0x5d8a9099L, 0x594b8d2eL, 0x5408abf7L, 0x50c9b640L,
0x4e8ee645L, 0x4a4ffbf2L, 0x470cdd2bL, 0x43cdc09cL,
0x7b827d21L, 0x7f436096L, 0x7200464fL, 0x76c15bf8L,
0x68860bfdL, 0x6c47164aL, 0x61043093L, 0x65c52d24L,
0x119b4be9L, 0x155a565eL, 0x18197087L, 0x1cd86d30L,
0x029f3d35L, 0x065e2082L, 0x0b1d065bL, 0x0fdc1becL,
0x3793a651L, 0x3352bbe6L, 0x3e119d3fL, 0x3ad08088L,
0x2497d08dL, 0x2056cd3aL, 0x2d15ebe3L, 0x29d4f654L,
0xc5a92679L, 0xc1683bceL, 0xcc2b1d17L, 0xc8ea00a0L,
0xd6ad50a5L, 0xd26c4d12L, 0xdf2f6bcbL, 0xdbee767cL,
0xe3a1cbc1L, 0xe760d676L, 0xea23f0afL, 0xeee2ed18L,
0xf0a5bd1dL, 0xf464a0aaL, 0xf9278673L, 0xfde69bc4L,
0x89b8fd09L, 0x8d79e0beL, 0x803ac667L, 0x84fbdbd0L,
0x9abc8bd5L, 0x9e7d9662L, 0x933eb0bbL, 0x97ffad0cL,
0xafb010b1L, 0xab710d06L, 0xa6322bdfL, 0xa2f33668L,
0xbcb4666dL, 0xb8757bdaL, 0xb5365d03L, 0xb1f740b4L
};
/*-------------------------------------------------------------*/
/*--- end crctable.c ---*/
/*-------------------------------------------------------------*/

652
thirdparty/bzip2/decompress.c vendored
View file

@ -1,652 +0,0 @@
/*-------------------------------------------------------------*/
/*--- Decompression machinery ---*/
/*--- decompress.c ---*/
/*-------------------------------------------------------------*/
/* ------------------------------------------------------------------
This file is part of bzip2/libbzip2, a program and library for
lossless, block-sorting data compression.
bzip2/libbzip2 version 1.1.0 of 6 September 2010
Copyright (C) 1996-2010 Julian Seward <jseward@acm.org>
Please read the WARNING, DISCLAIMER and PATENTS sections in the
README file.
This program is released under the terms of the license contained
in the file LICENSE.
------------------------------------------------------------------ */
#include "bzlib_private.h"
/*---------------------------------------------------*/
static
void makeMaps_d ( DState* s )
{
Int32 i;
s->nInUse = 0;
for (i = 0; i < 256; i++)
if (s->inUse[i]) {
s->seqToUnseq[s->nInUse] = i;
s->nInUse++;
}
}
/*---------------------------------------------------*/
#define RETURN(rrr) \
{ retVal = rrr; goto save_state_and_return; };
#define GET_BITS(lll,vvv,nnn) \
case lll: s->state = lll; \
while (True) { \
if (s->bsLive >= nnn) { \
UInt32 v; \
v = (s->bsBuff >> \
(s->bsLive-nnn)) & ((1 << nnn)-1); \
s->bsLive -= nnn; \
vvv = v; \
break; \
} \
if (s->strm->avail_in == 0) RETURN(BZ_OK); \
s->bsBuff \
= (s->bsBuff << 8) | \
((UInt32) \
(*((UChar*)(s->strm->next_in)))); \
s->bsLive += 8; \
s->strm->next_in++; \
s->strm->avail_in--; \
s->strm->total_in_lo32++; \
if (s->strm->total_in_lo32 == 0) \
s->strm->total_in_hi32++; \
}
#define GET_UCHAR(lll,uuu) \
GET_BITS(lll,uuu,8)
#define GET_BIT(lll,uuu) \
GET_BITS(lll,uuu,1)
/*---------------------------------------------------*/
#define GET_MTF_VAL(label1,label2,lval) \
{ \
if (groupPos == 0) { \
groupNo++; \
if (groupNo >= nSelectors) \
RETURN(BZ_DATA_ERROR); \
groupPos = BZ_G_SIZE; \
gSel = s->selector[groupNo]; \
gMinlen = s->minLens[gSel]; \
gLimit = &(s->limit[gSel][0]); \
gPerm = &(s->perm[gSel][0]); \
gBase = &(s->base[gSel][0]); \
} \
groupPos--; \
zn = gMinlen; \
GET_BITS(label1, zvec, zn); \
while (1) { \
if (zn > 20 /* the longest code */) \
RETURN(BZ_DATA_ERROR); \
if (zvec <= gLimit[zn]) break; \
zn++; \
GET_BIT(label2, zj); \
zvec = (zvec << 1) | zj; \
}; \
if (zvec - gBase[zn] < 0 \
|| zvec - gBase[zn] >= BZ_MAX_ALPHA_SIZE) \
RETURN(BZ_DATA_ERROR); \
lval = gPerm[zvec - gBase[zn]]; \
}
/*---------------------------------------------------*/
Int32 BZ2_decompress ( DState* s )
{
UChar uc;
Int32 retVal;
Int32 minLen, maxLen;
bz_stream* strm = s->strm;
/* stuff that needs to be saved/restored */
Int32 i;
Int32 j;
Int32 t;
Int32 alphaSize;
Int32 nGroups;
Int32 nSelectors;
Int32 EOB;
Int32 groupNo;
Int32 groupPos;
Int32 nextSym;
Int32 nblockMAX;
Int32 nblock;
Int32 es;
Int32 N;
Int32 curr;
Int32 zt;
Int32 zn;
Int32 zvec;
Int32 zj;
Int32 gSel;
Int32 gMinlen;
Int32* gLimit;
Int32* gBase;
Int32* gPerm;
if (s->state == BZ_X_MAGIC_1) {
/*initialise the save area*/
s->save_i = 0;
s->save_j = 0;
s->save_t = 0;
s->save_alphaSize = 0;
s->save_nGroups = 0;
s->save_nSelectors = 0;
s->save_EOB = 0;
s->save_groupNo = 0;
s->save_groupPos = 0;
s->save_nextSym = 0;
s->save_nblockMAX = 0;
s->save_nblock = 0;
s->save_es = 0;
s->save_N = 0;
s->save_curr = 0;
s->save_zt = 0;
s->save_zn = 0;
s->save_zvec = 0;
s->save_zj = 0;
s->save_gSel = 0;
s->save_gMinlen = 0;
s->save_gLimit = NULL;
s->save_gBase = NULL;
s->save_gPerm = NULL;
}
/*restore from the save area*/
i = s->save_i;
j = s->save_j;
t = s->save_t;
alphaSize = s->save_alphaSize;
nGroups = s->save_nGroups;
nSelectors = s->save_nSelectors;
EOB = s->save_EOB;
groupNo = s->save_groupNo;
groupPos = s->save_groupPos;
nextSym = s->save_nextSym;
nblockMAX = s->save_nblockMAX;
nblock = s->save_nblock;
es = s->save_es;
N = s->save_N;
curr = s->save_curr;
zt = s->save_zt;
zn = s->save_zn;
zvec = s->save_zvec;
zj = s->save_zj;
gSel = s->save_gSel;
gMinlen = s->save_gMinlen;
gLimit = s->save_gLimit;
gBase = s->save_gBase;
gPerm = s->save_gPerm;
retVal = BZ_OK;
switch (s->state) {
GET_UCHAR(BZ_X_MAGIC_1, uc);
if (uc != BZ_HDR_B) RETURN(BZ_DATA_ERROR_MAGIC);
GET_UCHAR(BZ_X_MAGIC_2, uc);
if (uc != BZ_HDR_Z) RETURN(BZ_DATA_ERROR_MAGIC);
GET_UCHAR(BZ_X_MAGIC_3, uc)
if (uc != BZ_HDR_h) RETURN(BZ_DATA_ERROR_MAGIC);
GET_BITS(BZ_X_MAGIC_4, s->blockSize100k, 8)
if (s->blockSize100k < (BZ_HDR_0 + 1) ||
s->blockSize100k > (BZ_HDR_0 + 9)) RETURN(BZ_DATA_ERROR_MAGIC);
s->blockSize100k -= BZ_HDR_0;
if (s->smallDecompress) {
s->ll16 = BZALLOC( s->blockSize100k * 100000 * sizeof(UInt16) );
s->ll4 = BZALLOC(
((1 + s->blockSize100k * 100000) >> 1) * sizeof(UChar)
);
if (s->ll16 == NULL || s->ll4 == NULL) RETURN(BZ_MEM_ERROR);
} else {
s->tt = BZALLOC( s->blockSize100k * 100000 * sizeof(Int32) );
if (s->tt == NULL) RETURN(BZ_MEM_ERROR);
}
GET_UCHAR(BZ_X_BLKHDR_1, uc);
if (uc == 0x17) goto endhdr_2;
if (uc != 0x31) RETURN(BZ_DATA_ERROR);
GET_UCHAR(BZ_X_BLKHDR_2, uc);
if (uc != 0x41) RETURN(BZ_DATA_ERROR);
GET_UCHAR(BZ_X_BLKHDR_3, uc);
if (uc != 0x59) RETURN(BZ_DATA_ERROR);
GET_UCHAR(BZ_X_BLKHDR_4, uc);
if (uc != 0x26) RETURN(BZ_DATA_ERROR);
GET_UCHAR(BZ_X_BLKHDR_5, uc);
if (uc != 0x53) RETURN(BZ_DATA_ERROR);
GET_UCHAR(BZ_X_BLKHDR_6, uc);
if (uc != 0x59) RETURN(BZ_DATA_ERROR);
s->currBlockNo++;
if (s->verbosity >= 2)
VPrintf1 ( "\n [%d: huff+mtf ", s->currBlockNo );
s->storedBlockCRC = 0;
GET_UCHAR(BZ_X_BCRC_1, uc);
s->storedBlockCRC = (s->storedBlockCRC << 8) | ((UInt32)uc);
GET_UCHAR(BZ_X_BCRC_2, uc);
s->storedBlockCRC = (s->storedBlockCRC << 8) | ((UInt32)uc);
GET_UCHAR(BZ_X_BCRC_3, uc);
s->storedBlockCRC = (s->storedBlockCRC << 8) | ((UInt32)uc);
GET_UCHAR(BZ_X_BCRC_4, uc);
s->storedBlockCRC = (s->storedBlockCRC << 8) | ((UInt32)uc);
GET_BITS(BZ_X_RANDBIT, s->blockRandomised, 1);
s->origPtr = 0;
GET_UCHAR(BZ_X_ORIGPTR_1, uc);
s->origPtr = (s->origPtr << 8) | ((Int32)uc);
GET_UCHAR(BZ_X_ORIGPTR_2, uc);
s->origPtr = (s->origPtr << 8) | ((Int32)uc);
GET_UCHAR(BZ_X_ORIGPTR_3, uc);
s->origPtr = (s->origPtr << 8) | ((Int32)uc);
if (s->origPtr < 0)
RETURN(BZ_DATA_ERROR);
if (s->origPtr > 10 + 100000*s->blockSize100k)
RETURN(BZ_DATA_ERROR);
/*--- Receive the mapping table ---*/
for (i = 0; i < 16; i++) {
GET_BIT(BZ_X_MAPPING_1, uc);
if (uc == 1)
s->inUse16[i] = True; else
s->inUse16[i] = False;
}
for (i = 0; i < 256; i++) s->inUse[i] = False;
for (i = 0; i < 16; i++)
if (s->inUse16[i])
for (j = 0; j < 16; j++) {
GET_BIT(BZ_X_MAPPING_2, uc);
if (uc == 1) s->inUse[i * 16 + j] = True;
}
makeMaps_d ( s );
if (s->nInUse == 0) RETURN(BZ_DATA_ERROR);
alphaSize = s->nInUse+2;
/*--- Now the selectors ---*/
GET_BITS(BZ_X_SELECTOR_1, nGroups, 3);
if (nGroups < 2 || nGroups > BZ_N_GROUPS) RETURN(BZ_DATA_ERROR);
GET_BITS(BZ_X_SELECTOR_2, nSelectors, 15);
if (nSelectors < 1) RETURN(BZ_DATA_ERROR);
for (i = 0; i < nSelectors; i++) {
j = 0;
while (True) {
GET_BIT(BZ_X_SELECTOR_3, uc);
if (uc == 0) break;
j++;
if (j >= nGroups) RETURN(BZ_DATA_ERROR);
}
/* Having more than BZ_MAX_SELECTORS doesn't make much sense
since they will never be used, but some implementations might
"round up" the number of selectors, so just ignore those. */
if (i < BZ_MAX_SELECTORS)
s->selectorMtf[i] = j;
}
if (nSelectors > BZ_MAX_SELECTORS)
nSelectors = BZ_MAX_SELECTORS;
/*--- Undo the MTF values for the selectors. ---*/
{
UChar pos[BZ_N_GROUPS], tmp, v;
for (v = 0; v < nGroups; v++) pos[v] = v;
for (i = 0; i < nSelectors; i++) {
v = s->selectorMtf[i];
tmp = pos[v];
while (v > 0) { pos[v] = pos[v-1]; v--; }
pos[0] = tmp;
s->selector[i] = tmp;
}
}
/*--- Now the coding tables ---*/
for (t = 0; t < nGroups; t++) {
GET_BITS(BZ_X_CODING_1, curr, 5);
for (i = 0; i < alphaSize; i++) {
while (True) {
if (curr < 1 || curr > 20) RETURN(BZ_DATA_ERROR);
GET_BIT(BZ_X_CODING_2, uc);
if (uc == 0) break;
GET_BIT(BZ_X_CODING_3, uc);
if (uc == 0) curr++; else curr--;
}
s->len[t][i] = curr;
}
}
/*--- Create the Huffman decoding tables ---*/
for (t = 0; t < nGroups; t++) {
minLen = 32;
maxLen = 0;
for (i = 0; i < alphaSize; i++) {
if (s->len[t][i] > maxLen) maxLen = s->len[t][i];
if (s->len[t][i] < minLen) minLen = s->len[t][i];
}
BZ2_hbCreateDecodeTables (
&(s->limit[t][0]),
&(s->base[t][0]),
&(s->perm[t][0]),
&(s->len[t][0]),
minLen, maxLen, alphaSize
);
s->minLens[t] = minLen;
}
/*--- Now the MTF values ---*/
EOB = s->nInUse+1;
nblockMAX = 100000 * s->blockSize100k;
groupNo = -1;
groupPos = 0;
for (i = 0; i <= 255; i++) s->unzftab[i] = 0;
/*-- MTF init --*/
{
Int32 ii, jj, kk;
kk = MTFA_SIZE-1;
for (ii = 256 / MTFL_SIZE - 1; ii >= 0; ii--) {
for (jj = MTFL_SIZE-1; jj >= 0; jj--) {
s->mtfa[kk] = (UChar)(ii * MTFL_SIZE + jj);
kk--;
}
s->mtfbase[ii] = kk + 1;
}
}
/*-- end MTF init --*/
nblock = 0;
GET_MTF_VAL(BZ_X_MTF_1, BZ_X_MTF_2, nextSym);
while (True) {
if (nextSym == EOB) break;
if (nextSym == BZ_RUNA || nextSym == BZ_RUNB) {
es = -1;
N = 1;
do {
/* Check that N doesn't get too big, so that es doesn't
go negative. The maximum value that can be
RUNA/RUNB encoded is equal to the block size (post
the initial RLE), viz, 900k, so bounding N at 2
million should guard against overflow without
rejecting any legitimate inputs. */
if (N >= 2*1024*1024) RETURN(BZ_DATA_ERROR);
if (nextSym == BZ_RUNA) es = es + (0+1) * N; else
if (nextSym == BZ_RUNB) es = es + (1+1) * N;
N = N * 2;
GET_MTF_VAL(BZ_X_MTF_3, BZ_X_MTF_4, nextSym);
}
while (nextSym == BZ_RUNA || nextSym == BZ_RUNB);
es++;
uc = s->seqToUnseq[ s->mtfa[s->mtfbase[0]] ];
s->unzftab[uc] += es;
if (s->smallDecompress)
while (es > 0) {
if (nblock >= nblockMAX) RETURN(BZ_DATA_ERROR);
s->ll16[nblock] = (UInt16)uc;
nblock++;
es--;
}
else
while (es > 0) {
if (nblock >= nblockMAX) RETURN(BZ_DATA_ERROR);
s->tt[nblock] = (UInt32)uc;
nblock++;
es--;
};
continue;
} else {
if (nblock >= nblockMAX) RETURN(BZ_DATA_ERROR);
/*-- uc = MTF ( nextSym-1 ) --*/
{
Int32 ii, jj, kk, pp, lno, off;
UInt32 nn;
nn = (UInt32)(nextSym - 1);
if (nn < MTFL_SIZE) {
/* avoid general-case expense */
pp = s->mtfbase[0];
uc = s->mtfa[pp+nn];
while (nn > 3) {
Int32 z = pp+nn;
s->mtfa[(z) ] = s->mtfa[(z)-1];
s->mtfa[(z)-1] = s->mtfa[(z)-2];
s->mtfa[(z)-2] = s->mtfa[(z)-3];
s->mtfa[(z)-3] = s->mtfa[(z)-4];
nn -= 4;
}
while (nn > 0) {
s->mtfa[(pp+nn)] = s->mtfa[(pp+nn)-1]; nn--;
};
s->mtfa[pp] = uc;
} else {
/* general case */
lno = nn / MTFL_SIZE;
off = nn % MTFL_SIZE;
pp = s->mtfbase[lno] + off;
uc = s->mtfa[pp];
while (pp > s->mtfbase[lno]) {
s->mtfa[pp] = s->mtfa[pp-1]; pp--;
};
s->mtfbase[lno]++;
while (lno > 0) {
s->mtfbase[lno]--;
s->mtfa[s->mtfbase[lno]]
= s->mtfa[s->mtfbase[lno-1] + MTFL_SIZE - 1];
lno--;
}
s->mtfbase[0]--;
s->mtfa[s->mtfbase[0]] = uc;
if (s->mtfbase[0] == 0) {
kk = MTFA_SIZE-1;
for (ii = 256 / MTFL_SIZE-1; ii >= 0; ii--) {
for (jj = MTFL_SIZE-1; jj >= 0; jj--) {
s->mtfa[kk] = s->mtfa[s->mtfbase[ii] + jj];
kk--;
}
s->mtfbase[ii] = kk + 1;
}
}
}
}
/*-- end uc = MTF ( nextSym-1 ) --*/
s->unzftab[s->seqToUnseq[uc]]++;
if (s->smallDecompress)
s->ll16[nblock] = (UInt16)(s->seqToUnseq[uc]); else
s->tt[nblock] = (UInt32)(s->seqToUnseq[uc]);
nblock++;
GET_MTF_VAL(BZ_X_MTF_5, BZ_X_MTF_6, nextSym);
continue;
}
}
/* Now we know what nblock is, we can do a better sanity
check on s->origPtr.
*/
if (s->origPtr < 0 || s->origPtr >= nblock)
RETURN(BZ_DATA_ERROR);
/*-- Set up cftab to facilitate generation of T^(-1) --*/
/* Check: unzftab entries in range. */
for (i = 0; i <= 255; i++) {
if (s->unzftab[i] < 0 || s->unzftab[i] > nblock)
RETURN(BZ_DATA_ERROR);
}
/* Actually generate cftab. */
s->cftab[0] = 0;
for (i = 1; i <= 256; i++) s->cftab[i] = s->unzftab[i-1];
for (i = 1; i <= 256; i++) s->cftab[i] += s->cftab[i-1];
/* Check: cftab entries in range. */
for (i = 0; i <= 256; i++) {
if (s->cftab[i] < 0 || s->cftab[i] > nblock) {
/* s->cftab[i] can legitimately be == nblock */
RETURN(BZ_DATA_ERROR);
}
}
/* Check: cftab entries non-descending. */
for (i = 1; i <= 256; i++) {
if (s->cftab[i-1] > s->cftab[i]) {
RETURN(BZ_DATA_ERROR);
}
}
s->state_out_len = 0;
s->state_out_ch = 0;
BZ_INITIALISE_CRC ( s->calculatedBlockCRC );
s->state = BZ_X_OUTPUT;
if (s->verbosity >= 2) VPrintf0 ( "rt+rld" );
if (s->smallDecompress) {
/*-- Make a copy of cftab, used in generation of T --*/
for (i = 0; i <= 256; i++) s->cftabCopy[i] = s->cftab[i];
/*-- compute the T vector --*/
for (i = 0; i < nblock; i++) {
uc = (UChar)(s->ll16[i]);
SET_LL(i, s->cftabCopy[uc]);
s->cftabCopy[uc]++;
}
/*-- Compute T^(-1) by pointer reversal on T --*/
i = s->origPtr;
j = GET_LL(i);
do {
Int32 tmp = GET_LL(j);
SET_LL(j, i);
i = j;
j = tmp;
}
while (i != s->origPtr);
s->tPos = s->origPtr;
s->nblock_used = 0;
if (s->blockRandomised) {
BZ_RAND_INIT_MASK;
BZ_GET_SMALL(s->k0); s->nblock_used++;
BZ_RAND_UPD_MASK; s->k0 ^= BZ_RAND_MASK;
} else {
BZ_GET_SMALL(s->k0); s->nblock_used++;
}
} else {
/*-- compute the T^(-1) vector --*/
for (i = 0; i < nblock; i++) {
uc = (UChar)(s->tt[i] & 0xff);
s->tt[s->cftab[uc]] |= (i << 8);
s->cftab[uc]++;
}
s->tPos = s->tt[s->origPtr] >> 8;
s->nblock_used = 0;
if (s->blockRandomised) {
BZ_RAND_INIT_MASK;
BZ_GET_FAST(s->k0); s->nblock_used++;
BZ_RAND_UPD_MASK; s->k0 ^= BZ_RAND_MASK;
} else {
BZ_GET_FAST(s->k0); s->nblock_used++;
}
}
RETURN(BZ_OK);
endhdr_2:
GET_UCHAR(BZ_X_ENDHDR_2, uc);
if (uc != 0x72) RETURN(BZ_DATA_ERROR);
GET_UCHAR(BZ_X_ENDHDR_3, uc);
if (uc != 0x45) RETURN(BZ_DATA_ERROR);
GET_UCHAR(BZ_X_ENDHDR_4, uc);
if (uc != 0x38) RETURN(BZ_DATA_ERROR);
GET_UCHAR(BZ_X_ENDHDR_5, uc);
if (uc != 0x50) RETURN(BZ_DATA_ERROR);
GET_UCHAR(BZ_X_ENDHDR_6, uc);
if (uc != 0x90) RETURN(BZ_DATA_ERROR);
s->storedCombinedCRC = 0;
GET_UCHAR(BZ_X_CCRC_1, uc);
s->storedCombinedCRC = (s->storedCombinedCRC << 8) | ((UInt32)uc);
GET_UCHAR(BZ_X_CCRC_2, uc);
s->storedCombinedCRC = (s->storedCombinedCRC << 8) | ((UInt32)uc);
GET_UCHAR(BZ_X_CCRC_3, uc);
s->storedCombinedCRC = (s->storedCombinedCRC << 8) | ((UInt32)uc);
GET_UCHAR(BZ_X_CCRC_4, uc);
s->storedCombinedCRC = (s->storedCombinedCRC << 8) | ((UInt32)uc);
s->state = BZ_X_IDLE;
RETURN(BZ_STREAM_END);
default: AssertH ( False, 4001 );
}
AssertH ( False, 4002 );
save_state_and_return:
s->save_i = i;
s->save_j = j;
s->save_t = t;
s->save_alphaSize = alphaSize;
s->save_nGroups = nGroups;
s->save_nSelectors = nSelectors;
s->save_EOB = EOB;
s->save_groupNo = groupNo;
s->save_groupPos = groupPos;
s->save_nextSym = nextSym;
s->save_nblockMAX = nblockMAX;
s->save_nblock = nblock;
s->save_es = es;
s->save_N = N;
s->save_curr = curr;
s->save_zt = zt;
s->save_zn = zn;
s->save_zvec = zvec;
s->save_zj = zj;
s->save_gSel = gSel;
s->save_gMinlen = gMinlen;
s->save_gLimit = gLimit;
s->save_gBase = gBase;
s->save_gPerm = gPerm;
return retVal;
}
/*-------------------------------------------------------------*/
/*--- end decompress.c ---*/
/*-------------------------------------------------------------*/

175
thirdparty/bzip2/dlltest.c vendored
View file

@ -1,175 +0,0 @@
/*
minibz2
libbz2.dll test program.
by Yoshioka Tsuneo (tsuneo@rr.iij4u.or.jp)
This file is Public Domain. Welcome any email to me.
usage: minibz2 [-d] [-{1,2,..9}] [[srcfilename] destfilename]
*/
#define BZ_IMPORT
#include <stdio.h>
#include <stdlib.h>
#include "bzlib.h"
#ifdef _WIN32
#include <io.h>
#endif
#ifdef _WIN32
#define BZ2_LIBNAME "libbz2-1.0.2.DLL"
#include <windows.h>
static int BZ2DLLLoaded = 0;
static HINSTANCE BZ2DLLhLib;
int BZ2DLLLoadLibrary(void)
{
HINSTANCE hLib;
if(BZ2DLLLoaded==1){return 0;}
hLib=LoadLibrary(BZ2_LIBNAME);
if(hLib == NULL){
fprintf(stderr,"Can't load %s\n",BZ2_LIBNAME);
return -1;
}
BZ2_bzlibVersion=GetProcAddress(hLib,"BZ2_bzlibVersion");
BZ2_bzopen=GetProcAddress(hLib,"BZ2_bzopen");
BZ2_bzdopen=GetProcAddress(hLib,"BZ2_bzdopen");
BZ2_bzread=GetProcAddress(hLib,"BZ2_bzread");
BZ2_bzwrite=GetProcAddress(hLib,"BZ2_bzwrite");
BZ2_bzflush=GetProcAddress(hLib,"BZ2_bzflush");
BZ2_bzclose=GetProcAddress(hLib,"BZ2_bzclose");
BZ2_bzerror=GetProcAddress(hLib,"BZ2_bzerror");
if (!BZ2_bzlibVersion || !BZ2_bzopen || !BZ2_bzdopen
|| !BZ2_bzread || !BZ2_bzwrite || !BZ2_bzflush
|| !BZ2_bzclose || !BZ2_bzerror) {
fprintf(stderr,"GetProcAddress failed.\n");
return -1;
}
BZ2DLLLoaded=1;
BZ2DLLhLib=hLib;
return 0;
}
int BZ2DLLFreeLibrary(void)
{
if(BZ2DLLLoaded==0){return 0;}
FreeLibrary(BZ2DLLhLib);
BZ2DLLLoaded=0;
}
#endif /* WIN32 */
void usage(void)
{
puts("usage: minibz2 [-d] [-{1,2,..9}] [[srcfilename] destfilename]");
}
int main(int argc,char *argv[])
{
int decompress = 0;
int level = 9;
char *fn_r = NULL;
char *fn_w = NULL;
#ifdef _WIN32
if(BZ2DLLLoadLibrary()<0){
fprintf(stderr,"Loading of %s failed. Giving up.\n", BZ2_LIBNAME);
exit(1);
}
printf("Loading of %s succeeded. Library version is %s.\n",
BZ2_LIBNAME, BZ2_bzlibVersion() );
#endif
while(++argv,--argc){
if(**argv =='-' || **argv=='/'){
char *p;
for(p=*argv+1;*p;p++){
if(*p=='d'){
decompress = 1;
}else if('1'<=*p && *p<='9'){
level = *p - '0';
}else{
usage();
exit(1);
}
}
}else{
break;
}
}
if(argc>=1){
fn_r = *argv;
argc--;argv++;
}else{
fn_r = NULL;
}
if(argc>=1){
fn_w = *argv;
argc--;argv++;
}else{
fn_w = NULL;
}
{
int len;
char buff[0x1000];
char mode[10];
if(decompress){
BZFILE *BZ2fp_r = NULL;
FILE *fp_w = NULL;
if(fn_w){
if((fp_w = fopen(fn_w,"wb"))==NULL){
printf("can't open [%s]\n",fn_w);
perror("reason:");
exit(1);
}
}else{
fp_w = stdout;
}
if((fn_r == NULL && (BZ2fp_r = BZ2_bzdopen(fileno(stdin),"rb"))==NULL)
|| (fn_r != NULL && (BZ2fp_r = BZ2_bzopen(fn_r,"rb"))==NULL)){
printf("can't bz2openstream\n");
exit(1);
}
while((len=BZ2_bzread(BZ2fp_r,buff,0x1000))>0){
fwrite(buff,1,len,fp_w);
}
BZ2_bzclose(BZ2fp_r);
if(fp_w != stdout) fclose(fp_w);
}else{
BZFILE *BZ2fp_w = NULL;
FILE *fp_r = NULL;
if(fn_r){
if((fp_r = fopen(fn_r,"rb"))==NULL){
printf("can't open [%s]\n",fn_r);
perror("reason:");
exit(1);
}
}else{
fp_r = stdin;
}
mode[0]='w';
mode[1] = '0' + level;
mode[2] = '\0';
if((fn_w == NULL && (BZ2fp_w = BZ2_bzdopen(fileno(stdout),mode))==NULL)
|| (fn_w !=NULL && (BZ2fp_w = BZ2_bzopen(fn_w,mode))==NULL)){
printf("can't bz2openstream\n");
exit(1);
}
while((len=fread(buff,1,0x1000,fp_r))>0){
BZ2_bzwrite(BZ2fp_w,buff,len);
}
BZ2_bzclose(BZ2fp_w);
if(fp_r!=stdin)fclose(fp_r);
}
}
#ifdef _WIN32
BZ2DLLFreeLibrary();
#endif
return 0;
}

21
thirdparty/bzip2/docs/CMakeLists.txt vendored
View file

@ -1,21 +0,0 @@
set(DOC_TYPES pdf ps html)
find_program(PROG_SH sh)
if(NOT PROG_SH)
message(FATAL_ERROR "Generator not found!")
endif()
foreach(t IN LISTS DOC_TYPES)
add_custom_command(
OUTPUT
manual.${t}
COMMAND
${PROG_SH} xmlproc.sh -${t} manual.${t}
)
install(
FILES
${CMAKE_CURRENT_BINARY_DIR}/manual.${t}
DESTINATION
${CMAKE_INSTALL_PREFIX}/docs)
endforeach()

14
thirdparty/bzip2/docs/RELEASING.md vendored
View file

@ -1,14 +0,0 @@
# Release Checklist
* [ ] Increment package version at the top of [meson.build](../meson.build) and
[CMakeLists.txt](../CMakeLists.txt)
* [ ] Increment library revision in [meson.build](../meson.build) and
[CMakeLists.txt](../CMakeLists.txt)
* [ ] If interfaces were added, changed, or deleted, adjust per
[meson.build](../meson.build) and [CMakeLists.txt](../CMakeLists.txt).
See the GNU libtool versioning rules for library revision numbering advice:
http://www.gnu.org/software/libtool/manual/html_node/Updating-version-info.html
* [ ] On release day, create a new `release/*` branch and create a release tag.

39
thirdparty/bzip2/docs/bz-common.xsl vendored
View file

@ -1,39 +0,0 @@
<?xml version="1.0"?> <!-- -*- sgml -*- -->
<xsl:stylesheet
xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
<!-- we like '1.2 Title' -->
<xsl:param name="section.autolabel" select="'1'"/>
<xsl:param name="section.label.includes.component.label" select="'1'"/>
<!-- Do not put 'Chapter' at the start of eg 'Chapter 1. Doing This' -->
<xsl:param name="local.l10n.xml" select="document('')"/>
<l:i18n xmlns:l="http://docbook.sourceforge.net/xmlns/l10n/1.0">
<l:l10n language="en">
<l:context name="title-numbered">
<l:template name="chapter" text="%n.&#160;%t"/>
</l:context>
</l:l10n>
</l:i18n>
<!-- don't generate sub-tocs for qanda sets -->
<xsl:param name="generate.toc">
set toc,title
book toc,title,figure,table,example,equation
chapter toc,title
section toc
sect1 toc
sect2 toc
sect3 toc
sect4 nop
sect5 nop
qandaset toc
qandadiv nop
appendix toc,title
article/appendix nop
article toc,title
preface toc,title
reference toc,title
</xsl:param>
</xsl:stylesheet>

276
thirdparty/bzip2/docs/bz-fo.xsl vendored
View file

@ -1,276 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?> <!-- -*- sgml -*- -->
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:fo="http://www.w3.org/1999/XSL/Format" version="1.0">
<xsl:import href="http://docbook.sourceforge.net/release/xsl/current/fo/docbook.xsl"/>
<xsl:import href="bz-common.xsl"/>
<!-- set indent = yes while debugging, then change to NO -->
<xsl:output method="xml" indent="yes"/>
<!-- ensure only passivetex extensions are on -->
<xsl:param name="stylesheet.result.type" select="'fo'"/>
<!-- fo extensions: PDF bookmarks and index terms -->
<xsl:param name="use.extensions" select="'1'"/>
<xsl:param name="xep.extensions" select="0"/>
<xsl:param name="fop.extensions" select="0"/>
<xsl:param name="saxon.extensions" select="0"/>
<xsl:param name="passivetex.extensions" select="1"/>
<xsl:param name="tablecolumns.extension" select="'1'"/>
<!-- ensure we are using single sided -->
<xsl:param name="double.sided" select="'0'"/>
<!-- insert cross references to page numbers -->
<xsl:param name="insert.xref.page.number" select="1"/>
<!-- <?custom-pagebreak?> inserts a page break at this point -->
<xsl:template match="processing-instruction('custom-pagebreak')">
<fo:block break-before='page'/>
</xsl:template>
<!-- show links in color -->
<xsl:attribute-set name="xref.properties">
<xsl:attribute name="color">blue</xsl:attribute>
</xsl:attribute-set>
<!-- make pre listings indented a bit + a bg colour -->
<xsl:template match="programlisting | screen">
<fo:block start-indent="0.25in" wrap-option="no-wrap"
white-space-collapse="false" text-align="start"
font-family="monospace" background-color="#f2f2f9"
linefeed-treatment="preserve"
xsl:use-attribute-sets="normal.para.spacing">
<xsl:apply-templates/>
</fo:block>
</xsl:template>
<!-- make verbatim output prettier -->
<xsl:template match="literallayout">
<fo:block start-indent="0.25in" wrap-option="no-wrap"
white-space-collapse="false" text-align="start"
font-family="monospace" background-color="#edf7f4"
linefeed-treatment="preserve"
space-before="0em" space-after="0em">
<xsl:apply-templates/>
</fo:block>
</xsl:template>
<!-- workaround bug in passivetex fo output for itemizedlist -->
<xsl:template match="itemizedlist/listitem">
<xsl:variable name="id">
<xsl:call-template name="object.id"/></xsl:variable>
<xsl:variable name="itemsymbol">
<xsl:call-template name="list.itemsymbol">
<xsl:with-param name="node" select="parent::itemizedlist"/>
</xsl:call-template>
</xsl:variable>
<xsl:variable name="item.contents">
<fo:list-item-label end-indent="label-end()">
<fo:block>
<xsl:choose>
<xsl:when test="$itemsymbol='disc'">&#x2022;</xsl:when>
<xsl:when test="$itemsymbol='bullet'">&#x2022;</xsl:when>
<xsl:otherwise>&#x2022;</xsl:otherwise>
</xsl:choose>
</fo:block>
</fo:list-item-label>
<fo:list-item-body start-indent="body-start()">
<xsl:apply-templates/> <!-- removed extra block wrapper -->
</fo:list-item-body>
</xsl:variable>
<xsl:choose>
<xsl:when test="parent::*/@spacing = 'compact'">
<fo:list-item id="{$id}"
xsl:use-attribute-sets="compact.list.item.spacing">
<xsl:copy-of select="$item.contents"/>
</fo:list-item>
</xsl:when>
<xsl:otherwise>
<fo:list-item id="{$id}" xsl:use-attribute-sets="list.item.spacing">
<xsl:copy-of select="$item.contents"/>
</fo:list-item>
</xsl:otherwise>
</xsl:choose>
</xsl:template>
<!-- workaround bug in passivetex fo output for orderedlist -->
<xsl:template match="orderedlist/listitem">
<xsl:variable name="id">
<xsl:call-template name="object.id"/></xsl:variable>
<xsl:variable name="item.contents">
<fo:list-item-label end-indent="label-end()">
<fo:block>
<xsl:apply-templates select="." mode="item-number"/>
</fo:block>
</fo:list-item-label>
<fo:list-item-body start-indent="body-start()">
<xsl:apply-templates/> <!-- removed extra block wrapper -->
</fo:list-item-body>
</xsl:variable>
<xsl:choose>
<xsl:when test="parent::*/@spacing = 'compact'">
<fo:list-item id="{$id}"
xsl:use-attribute-sets="compact.list.item.spacing">
<xsl:copy-of select="$item.contents"/>
</fo:list-item>
</xsl:when>
<xsl:otherwise>
<fo:list-item id="{$id}" xsl:use-attribute-sets="list.item.spacing">
<xsl:copy-of select="$item.contents"/>
</fo:list-item>
</xsl:otherwise>
</xsl:choose>
</xsl:template>
<!-- workaround bug in passivetex fo output for variablelist -->
<xsl:param name="variablelist.as.blocks" select="1"/>
<xsl:template match="varlistentry" mode="vl.as.blocks">
<xsl:variable name="id">
<xsl:call-template name="object.id"/></xsl:variable>
<fo:block id="{$id}" xsl:use-attribute-sets="list.item.spacing"
keep-together.within-column="always"
keep-with-next.within-column="always">
<xsl:apply-templates select="term"/>
</fo:block>
<fo:block start-indent="0.5in" end-indent="0in"
space-after.minimum="0.2em"
space-after.optimum="0.4em"
space-after.maximum="0.6em">
<fo:block>
<xsl:apply-templates select="listitem"/>
</fo:block>
</fo:block>
</xsl:template>
<!-- workaround bug in footers: force right-align w/two 80|30 cols -->
<xsl:template name="footer.table">
<xsl:param name="pageclass" select="''"/>
<xsl:param name="sequence" select="''"/>
<xsl:param name="gentext-key" select="''"/>
<xsl:choose>
<xsl:when test="$pageclass = 'index'">
<xsl:attribute name="margin-left">0pt</xsl:attribute>
</xsl:when>
</xsl:choose>
<xsl:variable name="candidate">
<fo:table table-layout="fixed" width="100%">
<fo:table-column column-number="1" column-width="80%"/>
<fo:table-column column-number="2" column-width="20%"/>
<fo:table-body>
<fo:table-row height="14pt">
<fo:table-cell text-align="left" display-align="after">
<xsl:attribute name="relative-align">baseline</xsl:attribute>
<fo:block>
<fo:block> </fo:block><!-- empty cell -->
</fo:block>
</fo:table-cell>
<fo:table-cell text-align="center" display-align="after">
<xsl:attribute name="relative-align">baseline</xsl:attribute>
<fo:block>
<xsl:call-template name="footer.content">
<xsl:with-param name="pageclass" select="$pageclass"/>
<xsl:with-param name="sequence" select="$sequence"/>
<xsl:with-param name="position" select="'center'"/>
<xsl:with-param name="gentext-key" select="$gentext-key"/>
</xsl:call-template>
</fo:block>
</fo:table-cell>
</fo:table-row>
</fo:table-body>
</fo:table>
</xsl:variable>
<!-- Really output a footer? -->
<xsl:choose>
<xsl:when test="$pageclass='titlepage' and $gentext-key='book'
and $sequence='first'">
<!-- no, book titlepages have no footers at all -->
</xsl:when>
<xsl:when test="$sequence = 'blank' and $footers.on.blank.pages = 0">
<!-- no output -->
</xsl:when>
<xsl:otherwise>
<xsl:copy-of select="$candidate"/>
</xsl:otherwise>
</xsl:choose>
</xsl:template>
<!-- fix bug in headers: force right-align w/two 40|60 cols -->
<xsl:template name="header.table">
<xsl:param name="pageclass" select="''"/>
<xsl:param name="sequence" select="''"/>
<xsl:param name="gentext-key" select="''"/>
<xsl:choose>
<xsl:when test="$pageclass = 'index'">
<xsl:attribute name="margin-left">0pt</xsl:attribute>
</xsl:when>
</xsl:choose>
<xsl:variable name="candidate">
<fo:table table-layout="fixed" width="100%">
<xsl:call-template name="head.sep.rule">
<xsl:with-param name="pageclass" select="$pageclass"/>
<xsl:with-param name="sequence" select="$sequence"/>
<xsl:with-param name="gentext-key" select="$gentext-key"/>
</xsl:call-template>
<fo:table-column column-number="1" column-width="40%"/>
<fo:table-column column-number="2" column-width="60%"/>
<fo:table-body>
<fo:table-row height="14pt">
<fo:table-cell text-align="left" display-align="before">
<xsl:attribute name="relative-align">baseline</xsl:attribute>
<fo:block>
<fo:block> </fo:block><!-- empty cell -->
</fo:block>
</fo:table-cell>
<fo:table-cell text-align="center" display-align="before">
<xsl:attribute name="relative-align">baseline</xsl:attribute>
<fo:block>
<xsl:call-template name="header.content">
<xsl:with-param name="pageclass" select="$pageclass"/>
<xsl:with-param name="sequence" select="$sequence"/>
<xsl:with-param name="position" select="'center'"/>
<xsl:with-param name="gentext-key" select="$gentext-key"/>
</xsl:call-template>
</fo:block>
</fo:table-cell>
</fo:table-row>
</fo:table-body>
</fo:table>
</xsl:variable>
<!-- Really output a header? -->
<xsl:choose>
<xsl:when test="$pageclass = 'titlepage' and $gentext-key = 'book'
and $sequence='first'">
<!-- no, book titlepages have no headers at all -->
</xsl:when>
<xsl:when test="$sequence = 'blank' and $headers.on.blank.pages = 0">
<!-- no output -->
</xsl:when>
<xsl:otherwise>
<xsl:copy-of select="$candidate"/>
</xsl:otherwise>
</xsl:choose>
</xsl:template>
<!-- Bug-fix for Suse 10 PassiveTex version -->
<!-- Precompute attribute values 'cos PassiveTex is too stupid: -->
<xsl:attribute-set name="component.title.properties">
<xsl:attribute name="keep-with-next.within-column">always</xsl:attribute>
<xsl:attribute name="space-before.optimum">
<xsl:value-of select="concat($body.font.master, 'pt')"/>
</xsl:attribute>
<xsl:attribute name="space-before.minimum">
<xsl:value-of select="$body.font.master * 0.8"/>
<xsl:text>pt</xsl:text>
</xsl:attribute>
<xsl:attribute name="space-before.maximum">
<xsl:value-of select="$body.font.master * 1.2"/>
<xsl:text>pt</xsl:text>
</xsl:attribute>
<xsl:attribute name="hyphenate">false</xsl:attribute>
</xsl:attribute-set>
</xsl:stylesheet>

23
thirdparty/bzip2/docs/bz-html.xsl vendored
View file

@ -1,23 +0,0 @@
<?xml version="1.0"?> <!-- -*- sgml -*- -->
<!DOCTYPE xsl:stylesheet [ <!ENTITY bz-css SYSTEM "./bzip.css"> ]>
<xsl:stylesheet
xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
<xsl:import href="http://docbook.sourceforge.net/release/xsl/current/html/docbook.xsl"/>
<xsl:import href="bz-common.xsl"/>
<!-- use UTF-8 encoding -->
<xsl:output method="html" encoding="UTF-8" indent="yes"/>
<!-- we include the css as link and directly when generating one large file -->
<xsl:template name="user.head.content">
<xsl:text disable-output-escaping="yes">
<![CDATA[<]]>link rel="stylesheet" type="text/css" href="bzip.css" />
</xsl:text>
<style type="text/css" media="screen">
<xsl:text>&bz-css;</xsl:text>
</style>
</xsl:template>
</xsl:stylesheet>

74
thirdparty/bzip2/docs/bzip.css vendored
View file

@ -1,74 +0,0 @@
/* Colours:
#74240f dark brown h1, h2, h3, h4
#336699 medium blue links
#339999 turquoise link hover colour
#202020 almost black general text
#761596 purple md5sum text
#626262 dark gray pre border
#eeeeee very light gray pre background
#f2f2f9 very light blue nav table background
#3366cc medium blue nav table border
*/
a, a:link, a:visited, a:active { color: #336699; }
a:hover { color: #339999; }
body { font: 80%/126% sans-serif; }
h1, h2, h3, h4 { color: #74240f; }
dt { color: #336699; font-weight: bold }
dd {
margin-left: 1.5em;
padding-bottom: 0.8em;
}
/* -- ruler -- */
div.hr_blue {
height: 3px;
background:#ffffff url("/images/hr_blue.png") repeat-x; }
div.hr_blue hr { display:none; }
/* release styles */
#release p { margin-top: 0.4em; }
#release .md5sum { color: #761596; }
/* ------ styles for docs|manuals|howto ------ */
/* -- lists -- */
ul {
margin: 0px 4px 16px 16px;
padding: 0px;
list-style: url("/images/li-blue.png");
}
ul li {
margin-bottom: 10px;
}
ul ul {
list-style-type: none;
list-style-image: none;
margin-left: 0px;
}
/* header / footer nav tables */
table.nav {
border: solid 1px #3366cc;
background: #f2f2f9;
background-color: #f2f2f9;
margin-bottom: 0.5em;
}
/* don't have underlined links in chunked nav menus */
table.nav a { text-decoration: none; }
table.nav a:hover { text-decoration: underline; }
table.nav td { font-size: 85%; }
code, tt, pre { font-size: 120%; }
code, tt { color: #761596; }
div.literallayout, pre.programlisting, pre.screen {
color: #000000;
padding: 0.5em;
background: #eeeeee;
border: 1px solid #626262;
background-color: #eeeeee;
margin: 4px 0px 4px 0px;
}

9
thirdparty/bzip2/docs/entities.xml vendored
View file

@ -1,9 +0,0 @@
<!-- misc. strings -->
<!ENTITY bz-url "https://gitlab.com/bzip2/bzip2">
<!ENTITY bz-email "jseward@acm.org">
<!ENTITY bz-lifespan "1996-2010">
<!ENTITY bz-version "1.0.6">
<!ENTITY bz-date "6 September 2010">
<!ENTITY manual-title "bzip2 Manual">

68
thirdparty/bzip2/docs/format.pl vendored
View file

@ -1,68 +0,0 @@
#!/usr/bin/perl -w
#
# ------------------------------------------------------------------
# This file is part of bzip2/libbzip2, a program and library for
# lossless, block-sorting data compression.
#
# bzip2/libbzip2 version 1.1.0 of 6 September 2010
# Copyright (C) 1996-2010 Julian Seward <jseward@acm.org>
#
# Please read the WARNING, DISCLAIMER and PATENTS sections in the
# README file.
#
# This program is released under the terms of the license contained
# in the file LICENSE.
# ------------------------------------------------------------------
#
use strict;
# get command line values:
if ( $#ARGV !=1 ) {
die "Usage: $0 xml_infile xml_outfile\n";
}
my $infile = shift;
# check infile exists
die "Can't find file \"$infile\""
unless -f $infile;
# check we can read infile
if (! -r $infile) {
die "Can't read input $infile\n";
}
# check we can open infile
open( INFILE,"<$infile" ) or
die "Can't input $infile $!";
#my $outfile = 'fmt-manual.xml';
my $outfile = shift;
#print "Infile: $infile, Outfile: $outfile\n";
# check we can write to outfile
open( OUTFILE,">$outfile" ) or
die "Can't output $outfile $! for writing";
my ($prev, $curr, $str);
$prev = ''; $curr = '';
while ( <INFILE> ) {
print OUTFILE $prev;
$prev = $curr;
$curr = $_;
$str = '';
if ( $prev =~ /<programlisting>$|<screen>$/ ) {
chomp $prev;
$curr = join( '', $prev, "<![CDATA[", $curr );
$prev = '';
next;
}
elsif ( $curr =~ /<\/programlisting>|<\/screen>/ ) {
chomp $prev;
$curr = join( '', $prev, "]]>", $curr );
$prev = '';
next;
}
}
print OUTFILE $curr;
close INFILE;
close OUTFILE;
exit;

2929
thirdparty/bzip2/docs/manual.xml vendored
View file

File diff suppressed because it is too large Load diff

17
thirdparty/bzip2/docs/meson.build vendored
View file

@ -1,17 +0,0 @@
build_docs = get_option('docs')
docs = find_program('xsltproc', required : build_docs).found()
docs = docs and find_program('perl', required : build_docs).found()
docs = docs and find_program('xmllint', required : build_docs).found()
docs = docs and find_program('grep', required : build_docs).found()
docs = docs and find_program('pdfxmltex', required : build_docs).found()
docs = docs and find_program('pdftops', required : build_docs).found()
prog_sh = find_program('sh', required : build_docs)
if docs and prog_sh.found()
foreach t : ['pdf', 'ps', 'html']
custom_target(
'manual.' + t,
command : [prog_sh, 'xmlproc.sh', '-' + t, '@OUTPUT@'],
output : 'manual.' + t,
)
endforeach
endif

114
thirdparty/bzip2/docs/xmlproc.sh vendored
View file

@ -1,114 +0,0 @@
#!/bin/bash
# see the README file for usage etc.
#
# ------------------------------------------------------------------
# This file is part of bzip2/libbzip2, a program and library for
# lossless, block-sorting data compression.
#
# bzip2/libbzip2 version 1.1.0 of 6 September 2010
# Copyright (C) 1996-2010 Julian Seward <jseward@acm.org>
#
# Please read the WARNING, DISCLAIMER and PATENTS sections in the
# README file.
#
# This program is released under the terms of the license contained
# in the file LICENSE.
# ----------------------------------------------------------------
usage() {
echo '';
echo 'Usage: xmlproc.sh -[option] <filename.xml>';
echo 'Specify a target from:';
echo '-v verify xml file conforms to dtd';
echo '-html output in html format (single file)';
echo '-ps output in postscript format';
echo '-pdf output in pdf format';
exit;
}
if test $# -ne 2; then
usage
fi
# assign the variable for the output type
action=$1; shift
# assign the output filename
xmlfile=$1; shift
# and check user input it correct
if !(test -f $xmlfile); then
echo "No such file: $xmlfile";
exit;
fi
# some other stuff we will use
OUT=output
xsl_fo=bz-fo.xsl
xsl_html=bz-html.xsl
basename=$xmlfile
basename=${basename//'.xml'/''}
fofile="${basename}.fo"
htmlfile="${basename}.html"
pdffile="${basename}.pdf"
psfile="${basename}.ps"
xmlfmtfile="${basename}.fmt"
# first process the xmlfile with CDATA tags
./format.pl $xmlfile $xmlfmtfile
# so the shell knows where the catalogs live
export XML_CATALOG_FILES=/etc/xml/catalog
# post-processing tidy up
cleanup() {
echo "Cleaning up: $@"
while [ $# != 0 ]
do
arg=$1; shift;
echo " deleting $arg";
rm $arg
done
}
case $action in
-v)
flags='--noout --xinclude --noblanks --postvalid'
dtd='--dtdvalid http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd'
xmllint $flags $dtd $xmlfmtfile 2> $OUT
grep -F 'error' $OUT
rm $OUT
;;
-html)
echo "Creating $htmlfile ..."
xsltproc --nonet --xinclude -o $htmlfile $xsl_html $xmlfmtfile
cleanup $xmlfmtfile
;;
-pdf)
echo "Creating $pdffile ..."
xsltproc --nonet --xinclude -o $fofile $xsl_fo $xmlfmtfile
pdfxmltex $fofile >$OUT </dev/null
pdfxmltex $fofile >$OUT </dev/null
pdfxmltex $fofile >$OUT </dev/null
cleanup $OUT $xmlfmtfile *.aux *.fo *.log *.out
;;
-ps)
echo "Creating $psfile ..."
xsltproc --nonet --xinclude -o $fofile $xsl_fo $xmlfmtfile
pdfxmltex $fofile >$OUT </dev/null
pdfxmltex $fofile >$OUT </dev/null
pdfxmltex $fofile >$OUT </dev/null
pdftops $pdffile $psfile
cleanup $OUT $xmlfmtfile $pdffile *.aux *.fo *.log *.out
# passivetex is broken, so we can't go this route yet.
# xmltex $fofile >$OUT </dev/null
# xmltex $fofile >$OUT </dev/null
# xmltex $fofile >$OUT </dev/null
# dvips -R -q -o bzip-manual.ps *.dvi
;;
*)
usage
;;
esac

205
thirdparty/bzip2/huffman.c vendored
View file

@ -1,205 +0,0 @@
/*-------------------------------------------------------------*/
/*--- Huffman coding low-level stuff ---*/
/*--- huffman.c ---*/
/*-------------------------------------------------------------*/
/* ------------------------------------------------------------------
This file is part of bzip2/libbzip2, a program and library for
lossless, block-sorting data compression.
bzip2/libbzip2 version 1.1.0 of 6 September 2010
Copyright (C) 1996-2010 Julian Seward <jseward@acm.org>
Please read the WARNING, DISCLAIMER and PATENTS sections in the
README file.
This program is released under the terms of the license contained
in the file LICENSE.
------------------------------------------------------------------ */
#include "bzlib_private.h"
/*---------------------------------------------------*/
#define WEIGHTOF(zz0) ((zz0) & 0xffffff00)
#define DEPTHOF(zz1) ((zz1) & 0x000000ff)
#define MYMAX(zz2,zz3) ((zz2) > (zz3) ? (zz2) : (zz3))
#define ADDWEIGHTS(zw1,zw2) \
(WEIGHTOF(zw1)+WEIGHTOF(zw2)) | \
(1 + MYMAX(DEPTHOF(zw1),DEPTHOF(zw2)))
#define UPHEAP(z) \
{ \
Int32 zz, tmp; \
zz = z; tmp = heap[zz]; \
while (weight[tmp] < weight[heap[zz >> 1]]) { \
heap[zz] = heap[zz >> 1]; \
zz >>= 1; \
} \
heap[zz] = tmp; \
}
#define DOWNHEAP(z) \
{ \
Int32 zz, yy, tmp; \
zz = z; tmp = heap[zz]; \
while (True) { \
yy = zz << 1; \
if (yy > nHeap) break; \
if (yy < nHeap && \
weight[heap[yy+1]] < weight[heap[yy]]) \
yy++; \
if (weight[tmp] < weight[heap[yy]]) break; \
heap[zz] = heap[yy]; \
zz = yy; \
} \
heap[zz] = tmp; \
}
/*---------------------------------------------------*/
void BZ2_hbMakeCodeLengths ( UChar *len,
Int32 *freq,
Int32 alphaSize,
Int32 maxLen )
{
/*--
Nodes and heap entries run from 1. Entry 0
for both the heap and nodes is a sentinel.
--*/
Int32 nNodes, nHeap, n1, n2, i, j, k;
Bool tooLong;
Int32 heap [ BZ_MAX_ALPHA_SIZE + 2 ];
Int32 weight [ BZ_MAX_ALPHA_SIZE * 2 ];
Int32 parent [ BZ_MAX_ALPHA_SIZE * 2 ];
for (i = 0; i < alphaSize; i++)
weight[i+1] = (freq[i] == 0 ? 1 : freq[i]) << 8;
while (True) {
nNodes = alphaSize;
nHeap = 0;
heap[0] = 0;
weight[0] = 0;
parent[0] = -2;
for (i = 1; i <= alphaSize; i++) {
parent[i] = -1;
nHeap++;
heap[nHeap] = i;
UPHEAP(nHeap);
}
AssertH( nHeap < (BZ_MAX_ALPHA_SIZE+2), 2001 );
while (nHeap > 1) {
n1 = heap[1]; heap[1] = heap[nHeap]; nHeap--; DOWNHEAP(1);
n2 = heap[1]; heap[1] = heap[nHeap]; nHeap--; DOWNHEAP(1);
nNodes++;
parent[n1] = parent[n2] = nNodes;
weight[nNodes] = ADDWEIGHTS(weight[n1], weight[n2]);
parent[nNodes] = -1;
nHeap++;
heap[nHeap] = nNodes;
UPHEAP(nHeap);
}
AssertH( nNodes < (BZ_MAX_ALPHA_SIZE * 2), 2002 );
tooLong = False;
for (i = 1; i <= alphaSize; i++) {
j = 0;
k = i;
while (parent[k] >= 0) { k = parent[k]; j++; }
len[i-1] = j;
if (j > maxLen) tooLong = True;
}
if (! tooLong) break;
/* 17 Oct 04: keep-going condition for the following loop used
to be 'i < alphaSize', which missed the last element,
theoretically leading to the possibility of the compressor
looping. However, this count-scaling step is only needed if
one of the generated Huffman code words is longer than
maxLen, which up to and including version 1.0.2 was 20 bits,
which is extremely unlikely. In version 1.0.3 maxLen was
changed to 17 bits, which has minimal effect on compression
ratio, but does mean this scaling step is used from time to
time, enough to verify that it works.
This means that bzip2-1.0.3 and later will only produce
Huffman codes with a maximum length of 17 bits. However, in
order to preserve backwards compatibility with bitstreams
produced by versions pre-1.0.3, the decompressor must still
handle lengths of up to 20. */
for (i = 1; i <= alphaSize; i++) {
j = weight[i] >> 8;
j = 1 + (j / 2);
weight[i] = j << 8;
}
}
}
/*---------------------------------------------------*/
void BZ2_hbAssignCodes ( Int32 *code,
UChar *length,
Int32 minLen,
Int32 maxLen,
Int32 alphaSize )
{
Int32 n, vec, i;
vec = 0;
for (n = minLen; n <= maxLen; n++) {
for (i = 0; i < alphaSize; i++)
if (length[i] == n) { code[i] = vec; vec++; };
vec <<= 1;
}
}
/*---------------------------------------------------*/
void BZ2_hbCreateDecodeTables ( Int32 *limit,
Int32 *base,
Int32 *perm,
UChar *length,
Int32 minLen,
Int32 maxLen,
Int32 alphaSize )
{
Int32 pp, i, j, vec;
pp = 0;
for (i = minLen; i <= maxLen; i++)
for (j = 0; j < alphaSize; j++)
if (length[j] == i) { perm[pp] = j; pp++; };
for (i = 0; i < BZ_MAX_CODE_LEN; i++) base[i] = 0;
for (i = 0; i < alphaSize; i++) base[length[i]+1]++;
for (i = 1; i < BZ_MAX_CODE_LEN; i++) base[i] += base[i-1];
for (i = 0; i < BZ_MAX_CODE_LEN; i++) limit[i] = 0;
vec = 0;
for (i = minLen; i <= maxLen; i++) {
vec += (base[i+1] - base[i]);
limit[i] = vec-1;
vec <<= 1;
}
for (i = minLen + 1; i <= maxLen; i++)
base[i] = ((limit[i-1] + 1) << 1) - base[i];
}
/*-------------------------------------------------------------*/
/*--- end huffman.c ---*/
/*-------------------------------------------------------------*/

41
thirdparty/bzip2/install_links.py vendored
View file

@ -1,41 +0,0 @@
#!/usr/bin/env python3
"""Create a symlink or a copy of an installed file."""
import argparse
import os
import shutil
def main():
parser = argparse.ArgumentParser()
parser.add_argument('bindir')
parser.add_argument('source')
parser.add_argument('dest', nargs='+')
parser.add_argument('--use-links', action='store_true')
args = parser.parse_args()
os.chdir(os.environ['MESON_INSTALL_DESTDIR_PREFIX'])
os.chdir(args.bindir)
# Windows doesn't really use symlinks, just copy in that case. Windows
# before vista (xp) doesn't have symlinks at all.
if args.use_links:
func = os.symlink
verb = 'Linking'
else:
func = shutil.copy
verb = 'Copying'
# at least os.symlink will fail if the destination already exists, just
# remove the dest if it already exists.
for dest in args.dest:
if os.path.exists(dest):
os.unlink(dest)
func(args.source, dest)
print('{} {} to {}'.format(verb, args.source, dest))
if __name__ == "__main__":
main()

26
thirdparty/bzip2/libbz2.def vendored
View file

@ -1,26 +0,0 @@
LIBRARY bz2-1
EXPORTS
BZ2_bzCompressInit
BZ2_bzCompress
BZ2_bzCompressEnd
BZ2_bzDecompressInit
BZ2_bzDecompress
BZ2_bzDecompressEnd
BZ2_bzReadOpen
BZ2_bzReadClose
BZ2_bzReadGetUnused
BZ2_bzRead
BZ2_bzWriteOpen
BZ2_bzWrite
BZ2_bzWriteClose
BZ2_bzWriteClose64
BZ2_bzBuffToBuffCompress
BZ2_bzBuffToBuffDecompress
BZ2_bzlibVersion
BZ2_bzopen
BZ2_bzdopen
BZ2_bzread
BZ2_bzwrite
BZ2_bzflush
BZ2_bzclose
BZ2_bzerror

8
thirdparty/bzip2/man/CMakeLists.txt vendored
View file

@ -1,8 +0,0 @@
set(MAN_FILES bzip2.1 bzgrep.1 bzdiff.1 bzmore.1)
foreach(m IN LISTS MAN_FILES)
install(
FILES
${CMAKE_CURRENT_SOURCE_DIR}/${m}
DESTINATION
${CMAKE_INSTALL_PREFIX}/man/man1)
endforeach()

47
thirdparty/bzip2/man/bzdiff.1 vendored
View file

@ -1,47 +0,0 @@
\"Shamelessly copied from zmore.1 by Philippe Troin <phil@fifi.org>
\"for Debian GNU/Linux
.TH BZDIFF 1
.SH NAME
bzcmp, bzdiff \- compare bzip2 compressed files
.SH SYNOPSIS
.B bzcmp
[ cmp_options ] file1
[ file2 ]
.br
.B bzdiff
[ diff_options ] file1
[ file2 ]
.SH DESCRIPTION
.I Bzcmp
and
.I bzdiff
are used to invoke the
.I cmp
or the
.I diff
program on bzip2 compressed files. All options specified are passed
directly to
.I cmp
or
.IR diff "."
If only 1 file is specified, then the files compared are
.I file1
and an uncompressed
.IR file1 ".bz2."
If two files are specified, then they are uncompressed if necessary and fed to
.I cmp
or
.IR diff "."
The exit status from
.I cmp
or
.I diff
is preserved.
.SH "SEE ALSO"
cmp(1), diff(1), bzmore(1), bzless(1), bzgrep(1), bzip2(1)
.SH BUGS
Messages from the
.I cmp
or
.I diff
programs refer to temporary filenames instead of those specified.

56
thirdparty/bzip2/man/bzgrep.1 vendored
View file

@ -1,56 +0,0 @@
\"Shamelessly copied from zmore.1 by Philippe Troin <phil@fifi.org>
\"for Debian GNU/Linux
.TH BZGREP 1
.SH NAME
bzgrep, bzfgrep, bzegrep \- search possibly bzip2 compressed files for a regular expression
.SH SYNOPSIS
.B bzgrep
[ grep options ]
.BI [\ -e\ ] " pattern"
.IR filename ".\|.\|."
.br
.B bzegrep
[ grep -E options ]
.BI [\ -e\ ] " pattern"
.IR filename ".\|.\|."
.br
.B bzfgrep
[ grep -F options ]
.BI [\ -e\ ] " pattern"
.IR filename ".\|.\|."
.SH DESCRIPTION
.IR Bzgrep
is used to invoke the
.I grep
on bzip2-compressed files. All options specified are passed directly to
.I grep.
If no file is specified, then the standard input is decompressed
if necessary and fed to grep.
Otherwise the given files are uncompressed if necessary and fed to
.I grep.
.PP
If
.I bzgrep
is invoked as
.I bzegrep
or
.I bzfgrep
then
.I grep -E
or
.I grep -F
is used instead of
.I grep.
If the GREP environment variable is set,
.I bzgrep
uses it as the
.I grep
program to be invoked. For example:
for sh: GREP="grep -F" bzgrep string files
for csh: (setenv GREP "grep -F"; bzgrep string files)
.SH AUTHOR
Charles Levert (charles@comm.polymtl.ca). Adapted to bzip2 by Philippe
Troin <phil@fifi.org> for Debian GNU/Linux.
.SH "SEE ALSO"
grep(1), bzdiff(1), bzmore(1), bzless(1), bzip2(1)

475
thirdparty/bzip2/man/bzip2.1 vendored
View file

@ -1,475 +0,0 @@
.TH bzip2 1
.SH NAME
bzip2, bunzip2 \- a block-sorting file compressor, v1.0.6
.br
bzcat \- decompresses files to stdout
.br
bzip2recover \- recovers data from damaged bzip2 files
.SH SYNOPSIS
.ll +8
.B bzip2
.RB [ " \-cdfkqstvzVL123456789 " ]
[
.I "filenames \&..."
]
.br
.B bzip2
.RB [ " \-h|\-\-help " ]
.ll -8
.br
.B bunzip2
.RB [ " \-fkvsVL " ]
[
.I "filenames \&..."
]
.br
.B bunzip2
.RB [ " \-h|\-\-help " ]
.br
.B bzcat
.RB [ " \-s " ]
[
.I "filenames \&..."
]
.br
.B bzcat
.RB [ " \-h|\-\-help " ]
.br
.B bzip2recover
.I "filename"
.SH DESCRIPTION
.I bzip2
compresses files using the Burrows-Wheeler block sorting
text compression algorithm, and Huffman coding. Compression is
generally considerably better than that achieved by more conventional
LZ77/LZ78-based compressors, and approaches the performance of the PPM
family of statistical compressors.
The command-line options are deliberately very similar to
those of
.I GNU gzip,
but they are not identical.
.I bzip2
expects a list of file names to accompany the
command-line flags. Each file is replaced by a compressed version of
itself, with the name "original_name.bz2".
Each compressed file
has the same modification date, permissions, and, when possible,
ownership as the corresponding original, so that these properties can
be correctly restored at decompression time. File name handling is
naive in the sense that there is no mechanism for preserving original
file names, permissions, ownerships or dates in filesystems which lack
these concepts, or have serious file name length restrictions, such as
MS-DOS.
.I bzip2
and
.I bunzip2
will by default not overwrite existing
files. If you want this to happen, specify the \-f flag.
If no file names are specified,
.I bzip2
compresses from standard
input to standard output. In this case,
.I bzip2
will decline to
write compressed output to a terminal, as this would be entirely
incomprehensible and therefore pointless.
.I bunzip2
(or
.I bzip2 \-d)
decompresses all
specified files. Files which were not created by
.I bzip2
will be detected and ignored, and a warning issued.
.I bzip2
attempts to guess the filename for the decompressed file
from that of the compressed file as follows:
filename.bz2 becomes filename
filename.bz becomes filename
filename.tbz2 becomes filename.tar
filename.tbz becomes filename.tar
anyothername becomes anyothername.out
If the file does not end in one of the recognised endings,
.I .bz2,
.I .bz,
.I .tbz2
or
.I .tbz,
.I bzip2
complains that it cannot
guess the name of the original file, and uses the original name
with
.I .out
appended.
As with compression, supplying no
filenames causes decompression from
standard input to standard output.
.I bunzip2
will correctly decompress a file which is the
concatenation of two or more compressed files. The result is the
concatenation of the corresponding uncompressed files. Integrity
testing (\-t)
of concatenated
compressed files is also supported.
You can also compress or decompress files to the standard output by
giving the \-c flag. Multiple files may be compressed and
decompressed like this. The resulting outputs are fed sequentially to
stdout. Compression of multiple files
in this manner generates a stream
containing multiple compressed file representations. Such a stream
can be decompressed correctly only by
.I bzip2
version 0.9.0 or
later. Earlier versions of
.I bzip2
will stop after decompressing
the first file in the stream.
.I bzcat
(or
.I bzip2 -dc)
decompresses all specified files to
the standard output.
.I bzip2
will read arguments from the environment variables
.I BZIP2
and
.I BZIP,
in that order, and will process them
before any arguments read from the command line. This gives a
convenient way to supply default arguments.
Compression is always performed, even if the compressed
file is slightly
larger than the original. Files of less than about one hundred bytes
tend to get larger, since the compression mechanism has a constant
overhead in the region of 50 bytes. Random data (including the output
of most file compressors) is coded at about 8.05 bits per byte, giving
an expansion of around 0.5%.
As a self-check for your protection,
.I bzip2
uses 32-bit CRCs to
make sure that the decompressed version of a file is identical to the
original. This guards against corruption of the compressed data, and
against undetected bugs in
.I bzip2
(hopefully very unlikely). The
chances of data corruption going undetected is microscopic, about one
chance in four billion for each file processed. Be aware, though, that
the check occurs upon decompression, so it can only tell you that
something is wrong. It can't help you
recover the original uncompressed
data. You can use
.I bzip2recover
to try to recover data from
damaged files.
Unlike
.I GNU gzip,
.I bzip2
will not create a series of
.I .bz2
suffixes even when using the
.I --force
option:
filename.bz2 does not become filename.bz2.bz2
Return values: 0 for a normal exit, 1 for environmental problems (file
not found, invalid flags, I/O errors, &c), 2 to indicate a corrupt
compressed file, 3 for an internal consistency error (eg, bug) which
caused
.I bzip2
to panic.
.SH OPTIONS
.TP
.B \-c --stdout
Compress or decompress to standard output.
.TP
.B \-d --decompress
Force decompression.
.I bzip2,
.I bunzip2
and
.I bzcat
are
really the same program, and the decision about what actions to take is
done on the basis of which name is used. This flag overrides that
mechanism, and forces
.I bzip2
to decompress.
.TP
.B \-z --compress
The complement to \-d: forces compression, regardless of the
invocation name.
.TP
.B \-t --test
Check integrity of the specified file(s), but don't decompress them.
This really performs a trial decompression and throws away the result.
.TP
.B \-f --force
Force overwrite of output files. Normally,
.I bzip2
will not overwrite
existing output files. Also forces
.I bzip2
to break hard links
to files, which it otherwise wouldn't do.
bzip2 normally declines to decompress files which don't have the
correct magic header bytes. If forced (-f), however, it will pass
such files through unmodified. This is how GNU gzip behaves.
.TP
.B \-k --keep
Keep (don't delete) input files during compression
or decompression.
.TP
.B \-s --small
Reduce memory usage, for compression, decompression and testing. Files
are decompressed and tested using a modified algorithm which only
requires 2.5 bytes per block byte. This means any file can be
decompressed in 2300\ k of memory, albeit at about half the normal speed.
During compression, \-s selects a block size of 200\ k, which limits
memory use to around the same figure, at the expense of your compression
ratio. In short, if your machine is low on memory (8 megabytes or
less), use \-s for everything. See MEMORY MANAGEMENT below.
.TP
.B \-q --quiet
Suppress non-essential warning messages. Messages pertaining to
I/O errors and other critical events will not be suppressed.
.TP
.B \-v --verbose
Verbose mode -- show the compression ratio for each file processed.
Further \-v's increase the verbosity level, spewing out lots of
information which is primarily of interest for diagnostic purposes.
.TP
.B \-h \-\-help
Print a help message and exit.
.TP
.B \-L --license -V --version
Display the software version, license terms and conditions.
.TP
.B \-1 (or \-\-fast) to \-9 (or \-\-best)
Set the block size to 100 k, 200 k ... 900 k when compressing. Has no
effect when decompressing. See MEMORY MANAGEMENT below.
The \-\-fast and \-\-best aliases are primarily for GNU gzip
compatibility. In particular, \-\-fast doesn't make things
significantly faster.
And \-\-best merely selects the default behaviour.
.TP
.B \--
Treats all subsequent arguments as file names, even if they start
with a dash. This is so you can handle files with names beginning
with a dash, for example: bzip2 \-- \-myfilename.
.TP
.B \--repetitive-fast --repetitive-best
These flags are redundant in versions 0.9.5 and above. They provided
some coarse control over the behaviour of the sorting algorithm in
earlier versions, which was sometimes useful. 0.9.5 and above have an
improved algorithm which renders these flags irrelevant.
.SH MEMORY MANAGEMENT
.I bzip2
compresses large files in blocks. The block size affects
both the compression ratio achieved, and the amount of memory needed for
compression and decompression. The flags \-1 through \-9
specify the block size to be 100,000 bytes through 900,000 bytes (the
default) respectively. At decompression time, the block size used for
compression is read from the header of the compressed file, and
.I bunzip2
then allocates itself just enough memory to decompress
the file. Since block sizes are stored in compressed files, it follows
that the flags \-1 to \-9 are irrelevant to and so ignored
during decompression.
Compression and decompression requirements,
in bytes, can be estimated as:
Compression: 400\ k + ( 8 x block size )
Decompression: 100\ k + ( 4 x block size ), or
100\ k + ( 2.5 x block size )
Larger block sizes give rapidly diminishing marginal returns. Most of
the compression comes from the first two or three hundred k of block
size, a fact worth bearing in mind when using
.I bzip2
on small machines.
It is also important to appreciate that the decompression memory
requirement is set at compression time by the choice of block size.
For files compressed with the default 900\ k block size,
.I bunzip2
will require about 3700 kbytes to decompress. To support decompression
of any file on a 4 megabyte machine,
.I bunzip2
has an option to
decompress using approximately half this amount of memory, about 2300
kbytes. Decompression speed is also halved, so you should use this
option only where necessary. The relevant flag is -s.
In general, try and use the largest block size memory constraints allow,
since that maximises the compression achieved. Compression and
decompression speed are virtually unaffected by block size.
Another significant point applies to files which fit in a single block
-- that means most files you'd encounter using a large block size. The
amount of real memory touched is proportional to the size of the file,
since the file is smaller than a block. For example, compressing a file
20,000 bytes long with the flag -9 will cause the compressor to
allocate around 7600\ k of memory, but only touch 400\ k + 20000 * 8 = 560
kbytes of it. Similarly, the decompressor will allocate 3700\ k but only
touch 100\ k + 20000 * 4 = 180 kbytes.
Here is a table which summarises the maximum memory usage for different
block sizes. Also recorded is the total compressed size for 14 files of
the Calgary Text Compression Corpus totalling 3,141,622 bytes. This
column gives some feel for how compression varies with block size.
These figures tend to understate the advantage of larger block sizes for
larger files, since the Corpus is dominated by smaller files.
Compress Decompress Decompress Corpus
Flag usage usage -s usage Size
-1 1200k 500k 350k 914704
-2 2000k 900k 600k 877703
-3 2800k 1300k 850k 860338
-4 3600k 1700k 1100k 846899
-5 4400k 2100k 1350k 845160
-6 5200k 2500k 1600k 838626
-7 6100k 2900k 1850k 834096
-8 6800k 3300k 2100k 828642
-9 7600k 3700k 2350k 828642
.SH RECOVERING DATA FROM DAMAGED FILES
.I bzip2
compresses files in blocks, usually 900\ kbytes long. Each
block is handled independently. If a media or transmission error causes
a multi-block .bz2
file to become damaged, it may be possible to
recover data from the undamaged blocks in the file.
The compressed representation of each block is delimited by a 48-bit
pattern, which makes it possible to find the block boundaries with
reasonable certainty. Each block also carries its own 32-bit CRC, so
damaged blocks can be distinguished from undamaged ones.
.I bzip2recover
is a simple program whose purpose is to search for
blocks in .bz2 files, and write each block out into its own .bz2
file. You can then use
.I bzip2
\-t
to test the
integrity of the resulting files, and decompress those which are
undamaged.
.I bzip2recover
takes a single argument, the name of the damaged file,
and writes a number of files "rec00001file.bz2",
"rec00002file.bz2", etc., containing the extracted blocks.
The output filenames are designed so that the use of
wildcards in subsequent processing -- for example,
"bzip2 -dc rec*file.bz2 > recovered_data" -- processes the files in
the correct order.
.I bzip2recover
should be of most use dealing with large .bz2
files, as these will contain many blocks. It is clearly
futile to use it on damaged single-block files, since a
damaged block cannot be recovered. If you wish to minimise
any potential data loss through media or transmission errors,
you might consider compressing with a smaller
block size.
.SH PERFORMANCE NOTES
The sorting phase of compression gathers together similar strings in the
file. Because of this, files containing very long runs of repeated
symbols, like "aabaabaabaab ...\&" (repeated several hundred times) may
compress more slowly than normal. Versions 0.9.5 and above fare much
better than previous versions in this respect. The ratio between
worst-case and average-case compression time is in the region of 10:1.
For previous versions, this figure was more like 100:1. You can use the
\-vvvv option to monitor progress in great detail, if you want.
Decompression speed is unaffected by these phenomena.
.I bzip2
usually allocates several megabytes of memory to operate
in, and then charges all over it in a fairly random fashion. This means
that performance, both for compressing and decompressing, is largely
determined by the speed at which your machine can service cache misses.
Because of this, small changes to the code to reduce the miss rate have
been observed to give disproportionately large performance improvements.
I imagine
.I bzip2
will perform best on machines with very large caches.
.SH CAVEATS
I/O error messages are not as helpful as they could be.
.I bzip2
tries hard to detect I/O errors and exit cleanly, but the details of
what the problem is sometimes seem rather misleading.
This manual page pertains to version 1.1.0 of
.I bzip2.
Compressed data created by this version is entirely forwards and
backwards compatible with the previous public releases, versions
0.1pl2, 0.9.0, 0.9.5, 1.0.0, 1.0.1, 1.0.2 and above, but with the following
exception: 0.9.0 and above can correctly decompress multiple
concatenated compressed files. 0.1pl2 cannot do this; it will stop
after decompressing just the first file in the stream.
.I bzip2recover
versions prior to 1.0.2 used 32-bit integers to represent
bit positions in compressed files, so they could not handle compressed
files more than 512 megabytes long. Versions 1.0.2 and above use
64-bit ints on some platforms which support them (GNU supported
targets, and Windows). To establish whether or not bzip2recover was
built with such a limitation, run it without arguments. In any event
you can build yourself an unlimited version if you can recompile it
with MaybeUInt64 set to be an unsigned 64-bit integer.
.SH AUTHOR
Julian Seward, jseward@acm.org.
https://gitlab.com/bzip2/bzip2
The ideas embodied in
.I bzip2
are due to (at least) the following
people: Michael Burrows and David Wheeler (for the block sorting
transformation), David Wheeler (again, for the Huffman coder), Peter
Fenwick (for the structured coding model in the original
.I bzip,
and many refinements), and Alistair Moffat, Radford Neal and Ian Witten
(for the arithmetic coder in the original
.I bzip).
I am much
indebted for their help, support and advice. See the manual in the
source distribution for pointers to sources of documentation. Christian
von Roques encouraged me to look for faster sorting algorithms, so as to
speed up compression. Bela Lubkin encouraged me to improve the
worst-case compression performance.
Donna Robinson XMLised the documentation.
The bz* scripts are derived from those of GNU gzip.
Many people sent patches, helped
with portability problems, lent machines, gave advice and were generally
helpful.

399
thirdparty/bzip2/man/bzip2.1.preformatted vendored
View file

@ -1,399 +0,0 @@
bzip2(1) bzip2(1)
NNAAMMEE
bzip2, bunzip2 − a block‐sorting file compressor, v1.0.6
bzcat − decompresses files to stdout
bzip2recover − recovers data from damaged bzip2 files
SSYYNNOOPPSSIISS
bbzziipp22 [ −−ccddffkkqqssttvvzzVVLL112233445566778899 ] [ _f_i_l_e_n_a_m_e_s _._._. ]
bbuunnzziipp22 [ −−ffkkvvssVVLL ] [ _f_i_l_e_n_a_m_e_s _._._. ]
bbzzccaatt [ −−ss ] [ _f_i_l_e_n_a_m_e_s _._._. ]
bbzziipp22rreeccoovveerr _f_i_l_e_n_a_m_e
DDEESSCCRRIIPPTTIIOONN
_b_z_i_p_2 compresses files using the Burrows‐Wheeler block
sorting text compression algorithm, and Huffman coding.
Compression is generally considerably better than that
achieved by more conventional LZ77/LZ78‐based compressors,
and approaches the performance of the PPM family of sta­
tistical compressors.
The command‐line options are deliberately very similar to
those of _G_N_U _g_z_i_p_, but they are not identical.
_b_z_i_p_2 expects a list of file names to accompany the com­
mand‐line flags. Each file is replaced by a compressed
version of itself, with the name "original_name.bz2".
Each compressed file has the same modification date, per­
missions, and, when possible, ownership as the correspond­
ing original, so that these properties can be correctly
restored at decompression time. File name handling is
naive in the sense that there is no mechanism for preserv­
ing original file names, permissions, ownerships or dates
in filesystems which lack these concepts, or have serious
file name length restrictions, such as MS‐DOS.
_b_z_i_p_2 and _b_u_n_z_i_p_2 will by default not overwrite existing
files. If you want this to happen, specify the −f flag.
If no file names are specified, _b_z_i_p_2 compresses from
standard input to standard output. In this case, _b_z_i_p_2
will decline to write compressed output to a terminal, as
this would be entirely incomprehensible and therefore
pointless.
_b_u_n_z_i_p_2 (or _b_z_i_p_2 _−_d_) decompresses all specified files.
Files which were not created by _b_z_i_p_2 will be detected and
ignored, and a warning issued. _b_z_i_p_2 attempts to guess
the filename for the decompressed file from that of the
compressed file as follows:
filename.bz2 becomes filename
filename.bz becomes filename
filename.tbz2 becomes filename.tar
filename.tbz becomes filename.tar
anyothername becomes anyothername.out
If the file does not end in one of the recognised endings,
_._b_z_2_, _._b_z_, _._t_b_z_2 or _._t_b_z_, _b_z_i_p_2 complains that it cannot
guess the name of the original file, and uses the original
name with _._o_u_t appended.
As with compression, supplying no filenames causes decom­
pression from standard input to standard output.
_b_u_n_z_i_p_2 will correctly decompress a file which is the con­
catenation of two or more compressed files. The result is
the concatenation of the corresponding uncompressed files.
Integrity testing (−t) of concatenated compressed files is
also supported.
You can also compress or decompress files to the standard
output by giving the −c flag. Multiple files may be com­
pressed and decompressed like this. The resulting outputs
are fed sequentially to stdout. Compression of multiple
files in this manner generates a stream containing multi­
ple compressed file representations. Such a stream can be
decompressed correctly only by _b_z_i_p_2 version 0.9.0 or
later. Earlier versions of _b_z_i_p_2 will stop after decom­
pressing the first file in the stream.
_b_z_c_a_t (or _b_z_i_p_2 _‐_d_c_) decompresses all specified files to
the standard output.
_b_z_i_p_2 will read arguments from the environment variables
_B_Z_I_P_2 and _B_Z_I_P_, in that order, and will process them
before any arguments read from the command line. This
gives a convenient way to supply default arguments.
Compression is always performed, even if the compressed
file is slightly larger than the original. Files of less
than about one hundred bytes tend to get larger, since the
compression mechanism has a constant overhead in the
region of 50 bytes. Random data (including the output of
most file compressors) is coded at about 8.05 bits per
byte, giving an expansion of around 0.5%.
As a self‐check for your protection, _b_z_i_p_2 uses 32‐bit
CRCs to make sure that the decompressed version of a file
is identical to the original. This guards against corrup­
tion of the compressed data, and against undetected bugs
in _b_z_i_p_2 (hopefully very unlikely). The chances of data
corruption going undetected is microscopic, about one
chance in four billion for each file processed. Be aware,
though, that the check occurs upon decompression, so it
can only tell you that something is wrong. It can’t help
you recover the original uncompressed data. You can use
_b_z_i_p_2_r_e_c_o_v_e_r to try to recover data from damaged files.
Return values: 0 for a normal exit, 1 for environmental
problems (file not found, invalid flags, I/O errors, &c),
2 to indicate a corrupt compressed file, 3 for an internal
consistency error (eg, bug) which caused _b_z_i_p_2 to panic.
OOPPTTIIOONNSS
−−cc ‐‐‐‐ssttddoouutt
Compress or decompress to standard output.
−−dd ‐‐‐‐ddeeccoommpprreessss
Force decompression. _b_z_i_p_2_, _b_u_n_z_i_p_2 and _b_z_c_a_t are
really the same program, and the decision about
what actions to take is done on the basis of which
name is used. This flag overrides that mechanism,
and forces _b_z_i_p_2 to decompress.
−−zz ‐‐‐‐ccoommpprreessss
The complement to −d: forces compression,
regardless of the invocation name.
−−tt ‐‐‐‐tteesstt
Check integrity of the specified file(s), but don’t
decompress them. This really performs a trial
decompression and throws away the result.
−−ff ‐‐‐‐ffoorrccee
Force overwrite of output files. Normally, _b_z_i_p_2
will not overwrite existing output files. Also
forces _b_z_i_p_2 to break hard links to files, which it
otherwise wouldn’t do.
bzip2 normally declines to decompress files which
don’t have the correct magic header bytes. If
forced (‐f), however, it will pass such files
through unmodified. This is how GNU gzip behaves.
−−kk ‐‐‐‐kkeeeepp
Keep (don’t delete) input files during compression
or decompression.
−−ss ‐‐‐‐ssmmaallll
Reduce memory usage, for compression, decompression
and testing. Files are decompressed and tested
using a modified algorithm which only requires 2.5
bytes per block byte. This means any file can be
decompressed in 2300k of memory, albeit at about
half the normal speed.
During compression, −s selects a block size of
200k, which limits memory use to around the same
figure, at the expense of your compression ratio.
In short, if your machine is low on memory (8
megabytes or less), use −s for everything. See
MEMORY MANAGEMENT below.
−−qq ‐‐‐‐qquuiieett
Suppress non‐essential warning messages. Messages
pertaining to I/O errors and other critical events
will not be suppressed.
−−vv ‐‐‐‐vveerrbboossee
Verbose mode ‐‐ show the compression ratio for each
file processed. Further −v’s increase the ver­
bosity level, spewing out lots of information which
is primarily of interest for diagnostic purposes.
−−LL ‐‐‐‐lliicceennssee ‐‐VV ‐‐‐‐vveerrssiioonn
Display the software version, license terms and
conditions.
−−11 ((oorr −−−−ffaasstt)) ttoo −−99 ((oorr −−−−bbeesstt))
Set the block size to 100 k, 200 k .. 900 k when
compressing. Has no effect when decompressing.
See MEMORY MANAGEMENT below. The −−fast and −−best
aliases are primarily for GNU gzip compatibility.
In particular, −−fast doesn’t make things signifi­
cantly faster. And −−best merely selects the
default behaviour.
−−‐‐ Treats all subsequent arguments as file names, even
if they start with a dash. This is so you can han­
dle files with names beginning with a dash, for
example: bzip2 −‐ −myfilename.
−−‐‐rreeppeettiittiivvee‐‐ffaasstt ‐‐‐‐rreeppeettiittiivvee‐‐bbeesstt
These flags are redundant in versions 0.9.5 and
above. They provided some coarse control over the
behaviour of the sorting algorithm in earlier ver­
sions, which was sometimes useful. 0.9.5 and above
have an improved algorithm which renders these
flags irrelevant.
MMEEMMOORRYY MMAANNAAGGEEMMEENNTT
_b_z_i_p_2 compresses large files in blocks. The block size
affects both the compression ratio achieved, and the
amount of memory needed for compression and decompression.
The flags −1 through −9 specify the block size to be
100,000 bytes through 900,000 bytes (the default) respec­
tively. At decompression time, the block size used for
compression is read from the header of the compressed
file, and _b_u_n_z_i_p_2 then allocates itself just enough memory
to decompress the file. Since block sizes are stored in
compressed files, it follows that the flags −1 to −9 are
irrelevant to and so ignored during decompression.
Compression and decompression requirements, in bytes, can
be estimated as:
Compression: 400k + ( 8 x block size )
Decompression: 100k + ( 4 x block size ), or
100k + ( 2.5 x block size )
Larger block sizes give rapidly diminishing marginal
returns. Most of the compression comes from the first two
or three hundred k of block size, a fact worth bearing in
mind when using _b_z_i_p_2 on small machines. It is also
important to appreciate that the decompression memory
requirement is set at compression time by the choice of
block size.
For files compressed with the default 900k block size,
_b_u_n_z_i_p_2 will require about 3700 kbytes to decompress. To
support decompression of any file on a 4 megabyte machine,
_b_u_n_z_i_p_2 has an option to decompress using approximately
half this amount of memory, about 2300 kbytes. Decompres­
sion speed is also halved, so you should use this option
only where necessary. The relevant flag is ‐s.
In general, try and use the largest block size memory con­
straints allow, since that maximises the compression
achieved. Compression and decompression speed are virtu­
ally unaffected by block size.
Another significant point applies to files which fit in a
single block ‐‐ that means most files you’d encounter
using a large block size. The amount of real memory
touched is proportional to the size of the file, since the
file is smaller than a block. For example, compressing a
file 20,000 bytes long with the flag ‐9 will cause the
compressor to allocate around 7600k of memory, but only
touch 400k + 20000 * 8 = 560 kbytes of it. Similarly, the
decompressor will allocate 3700k but only touch 100k +
20000 * 4 = 180 kbytes.
Here is a table which summarises the maximum memory usage
for different block sizes. Also recorded is the total
compressed size for 14 files of the Calgary Text Compres­
sion Corpus totalling 3,141,622 bytes. This column gives
some feel for how compression varies with block size.
These figures tend to understate the advantage of larger
block sizes for larger files, since the Corpus is domi­
nated by smaller files.
Compress Decompress Decompress Corpus
Flag usage usage ‐s usage Size
‐1 1200k 500k 350k 914704
‐2 2000k 900k 600k 877703
‐3 2800k 1300k 850k 860338
‐4 3600k 1700k 1100k 846899
‐5 4400k 2100k 1350k 845160
‐6 5200k 2500k 1600k 838626
‐7 6100k 2900k 1850k 834096
‐8 6800k 3300k 2100k 828642
‐9 7600k 3700k 2350k 828642
RREECCOOVVEERRIINNGG DDAATTAA FFRROOMM DDAAMMAAGGEEDD FFIILLEESS
_b_z_i_p_2 compresses files in blocks, usually 900kbytes long.
Each block is handled independently. If a media or trans­
mission error causes a multi‐block .bz2 file to become
damaged, it may be possible to recover data from the
undamaged blocks in the file.
The compressed representation of each block is delimited
by a 48‐bit pattern, which makes it possible to find the
block boundaries with reasonable certainty. Each block
also carries its own 32‐bit CRC, so damaged blocks can be
distinguished from undamaged ones.
_b_z_i_p_2_r_e_c_o_v_e_r is a simple program whose purpose is to
search for blocks in .bz2 files, and write each block out
into its own .bz2 file. You can then use _b_z_i_p_2 −t to test
the integrity of the resulting files, and decompress those
which are undamaged.
_b_z_i_p_2_r_e_c_o_v_e_r takes a single argument, the name of the dam­
aged file, and writes a number of files
"rec00001file.bz2", "rec00002file.bz2", etc, containing
the extracted blocks. The output filenames are
designed so that the use of wildcards in subsequent pro­
cessing ‐‐ for example, "bzip2 ‐dc rec*file.bz2 > recov­
ered_data" ‐‐ processes the files in the correct order.
_b_z_i_p_2_r_e_c_o_v_e_r should be of most use dealing with large .bz2
files, as these will contain many blocks. It is clearly
futile to use it on damaged single‐block files, since a
damaged block cannot be recovered. If you wish to min­
imise any potential data loss through media or transmis­
sion errors, you might consider compressing with a smaller
block size.
PPEERRFFOORRMMAANNCCEE NNOOTTEESS
The sorting phase of compression gathers together similar
strings in the file. Because of this, files containing
very long runs of repeated symbols, like "aabaabaabaab
..." (repeated several hundred times) may compress more
slowly than normal. Versions 0.9.5 and above fare much
better than previous versions in this respect. The ratio
between worst‐case and average‐case compression time is in
the region of 10:1. For previous versions, this figure
was more like 100:1. You can use the −vvvv option to mon­
itor progress in great detail, if you want.
Decompression speed is unaffected by these phenomena.
_b_z_i_p_2 usually allocates several megabytes of memory to
operate in, and then charges all over it in a fairly ran­
dom fashion. This means that performance, both for com­
pressing and decompressing, is largely determined by the
speed at which your machine can service cache misses.
Because of this, small changes to the code to reduce the
miss rate have been observed to give disproportionately
large performance improvements. I imagine _b_z_i_p_2 will per­
form best on machines with very large caches.
CCAAVVEEAATTSS
I/O error messages are not as helpful as they could be.
_b_z_i_p_2 tries hard to detect I/O errors and exit cleanly,
but the details of what the problem is sometimes seem
rather misleading.
This manual page pertains to version 1.1.0 of _b_z_i_p_2_. Com­
pressed data created by this version is entirely forwards
and backwards compatible with the previous public
releases, versions 0.1pl2, 0.9.0, 0.9.5, 1.0.0, 1.0.1,
1.0.2 and above, but with the following exception: 0.9.0
and above can correctly decompress multiple concatenated
compressed files. 0.1pl2 cannot do this; it will stop
after decompressing just the first file in the stream.
_b_z_i_p_2_r_e_c_o_v_e_r versions prior to 1.0.2 used 32‐bit integers
to represent bit positions in compressed files, so they
could not handle compressed files more than 512 megabytes
long. Versions 1.0.2 and above use 64‐bit ints on some
platforms which support them (GNU supported targets, and
Windows). To establish whether or not bzip2recover was
built with such a limitation, run it without arguments.
In any event you can build yourself an unlimited version
if you can recompile it with MaybeUInt64 set to be an
unsigned 64‐bit integer.
AAUUTTHHOORR
Julian Seward, jseward@acm.org.
https://gitlab.com/bzip2/bzip2
The ideas embodied in _b_z_i_p_2 are due to (at least) the fol­
lowing people: Michael Burrows and David Wheeler (for the
block sorting transformation), David Wheeler (again, for
the Huffman coder), Peter Fenwick (for the structured cod­
ing model in the original _b_z_i_p_, and many refinements), and
Alistair Moffat, Radford Neal and Ian Witten (for the
arithmetic coder in the original _b_z_i_p_)_. I am much
indebted for their help, support and advice. See the man­
ual in the source distribution for pointers to sources of
documentation. Christian von Roques encouraged me to look
for faster sorting algorithms, so as to speed up compres­
sion. Bela Lubkin encouraged me to improve the worst‐case
compression performance. Donna Robinson XMLised the docu­
mentation. The bz* scripts are derived from those of GNU
gzip. Many people sent patches, helped with portability
problems, lent machines, gave advice and were generally
helpful.
bzip2(1)

152
thirdparty/bzip2/man/bzmore.1 vendored
View file

@ -1,152 +0,0 @@
.\"Shamelessly copied from zmore.1 by Philippe Troin <phil@fifi.org>
.\"for Debian GNU/Linux
.TH BZMORE 1
.SH NAME
bzmore, bzless \- file perusal filter for crt viewing of bzip2 compressed text
.SH SYNOPSIS
.B bzmore
[ name ... ]
.br
.B bzless
[ name ... ]
.SH NOTE
In the following description,
.I bzless
and
.I less
can be used interchangeably with
.I bzmore
and
.I more.
.SH DESCRIPTION
.I Bzmore
is a filter which allows examination of compressed or plain text files
one screenful at a time on a soft-copy terminal.
.I bzmore
works on files compressed with
.I bzip2
and also on uncompressed files.
If a file does not exist,
.I bzmore
looks for a file of the same name with the addition of a .bz2 suffix.
.PP
.I Bzmore
normally pauses after each screenful, printing --More--
at the bottom of the screen.
If the user then types a carriage return, one more line is displayed.
If the user hits a space,
another screenful is displayed. Other possibilities are enumerated later.
.PP
.I Bzmore
looks in the file
.I /etc/termcap
to determine terminal characteristics,
and to determine the default window size.
On a terminal capable of displaying 24 lines,
the default window size is 22 lines.
Other sequences which may be typed when
.I bzmore
pauses, and their effects, are as follows (\fIi\fP is an optional integer
argument, defaulting to 1) :
.PP
.IP \fIi\|\fP<space>
display
.I i
more lines, (or another screenful if no argument is given)
.PP
.IP ^D
display 11 more lines (a ``scroll'').
If
.I i
is given, then the scroll size is set to \fIi\|\fP.
.PP
.IP d
same as ^D (control-D)
.PP
.IP \fIi\|\fPz
same as typing a space except that \fIi\|\fP, if present, becomes the new
window size. Note that the window size reverts back to the default at the
end of the current file.
.PP
.IP \fIi\|\fPs
skip \fIi\|\fP lines and print a screenful of lines
.PP
.IP \fIi\|\fPf
skip \fIi\fP screenfuls and print a screenful of lines
.PP
.IP "q or Q"
quit reading the current file; go on to the next (if any)
.PP
.IP "e or q"
When the prompt --More--(Next file:
.IR file )
is printed, this command causes bzmore to exit.
.PP
.IP s
When the prompt --More--(Next file:
.IR file )
is printed, this command causes bzmore to skip the next file and continue.
.PP
.IP =
Display the current line number.
.PP
.IP \fIi\|\fP/expr
search for the \fIi\|\fP-th occurrence of the regular expression \fIexpr.\fP
If the pattern is not found,
.I bzmore
goes on to the next file (if any).
Otherwise, a screenful is displayed, starting two lines before the place
where the expression was found.
The user's erase and kill characters may be used to edit the regular
expression.
Erasing back past the first column cancels the search command.
.PP
.IP \fIi\|\fPn
search for the \fIi\|\fP-th occurrence of the last regular expression entered.
.PP
.IP !command
invoke a shell with \fIcommand\|\fP.
The character `!' in "command" are replaced with the
previous shell command. The sequence "\\!" is replaced by "!".
.PP
.IP ":q or :Q"
quit reading the current file; go on to the next (if any)
(same as q or Q).
.PP
.IP .
(dot) repeat the previous command.
.PP
The commands take effect immediately, i.e., it is not necessary to
type a carriage return.
Up to the time when the command character itself is given,
the user may hit the line kill character to cancel the numerical
argument being formed.
In addition, the user may hit the erase character to redisplay the
--More-- message.
.PP
At any time when output is being sent to the terminal, the user can
hit the quit key (normally control\-\\).
.I Bzmore
will stop sending output, and will display the usual --More--
prompt.
The user may then enter one of the above commands in the normal manner.
Unfortunately, some output is lost when this is done, due to the
fact that any characters waiting in the terminal's output queue
are flushed when the quit signal occurs.
.PP
The terminal is set to
.I noecho
mode by this program so that the output can be continuous.
What you type will thus not show on your terminal, except for the / and !
commands.
.PP
If the standard output is not a teletype, then
.I bzmore
acts just like
.I bzcat,
except that a header is printed before each file.
.SH FILES
.DT
/etc/termcap Terminal data base
.SH "SEE ALSO"
more(1), less(1), bzip2(1), bzdiff(1), bzgrep(1)

19
thirdparty/bzip2/man/meson.build vendored
View file

@ -1,19 +0,0 @@
install_man('bzip2.1', 'bzgrep.1', 'bzdiff.1', 'bzmore.1')
# Install copies of some of the man files
man1dir = join_paths(get_option('prefix'), get_option('mandir'), 'man1')
foreach m : [['bzip2.1', ['bunzip2.1', 'bzcat']],
['bzgrep.1', ['bzegrep.1', 'bzfgrep.1']],
['bzmore.1', ['bzless.1']],
['bzdiff.1', ['bzcmp.1']]]
_input = m[0]
foreach o : m[1]
configure_file(
input : _input,
output : o,
copy : true,
install : true,
install_dir : man1dir,
)
endforeach
endforeach

177
thirdparty/bzip2/meson.build vendored
View file

@ -1,177 +0,0 @@
project(
'bzip2',
['c'],
version : '1.1.0',
meson_version : '>= 0.56.0',
default_options : ['c_std=c89', 'warning_level=1'],
)
conf_data = configuration_data()
conf_data.set('BZ_VERSION', meson.project_version())
configure_file(
input: 'bz_version.h.in',
output: 'bz_version.h',
configuration: conf_data
)
cc = meson.get_compiler('c')
add_project_arguments(cc.get_supported_arguments([
# Please keep this list in sync with CMakeLists.txt
'-Wall',
'-Wextra',
'-Wmissing-prototypes',
'-Wstrict-prototypes',
'-Wmissing-declarations',
'-Wpointer-arith',
'-Wdeclaration-after-statement',
'-Wformat-security',
'-Wwrite-strings',
'-Wshadow',
'-Winline',
'-Wnested-externs',
'-Wfloat-equal',
'-Wundef',
'-Wendif-labels',
'-Wempty-body',
'-Wcast-align',
'-Wclobbered',
'-Wvla',
'-Wpragmas',
'-Wunreachable-code',
'-Waddress',
'-Wattributes',
'-Wdiv-by-zero',
'-Wshorten-64-to-32',
'-Wconversion',
'-Wextended-offsetof',
'-Wformat-nonliteral',
'-Wlanguage-extension-token',
'-Wmissing-field-initializers',
'-Wmissing-noreturn',
'-Wmissing-variable-declarations',
# '-Wpadded', # Not used because we cannot change public structs
'-Wsign-conversion',
# '-Wswitch-enum', # Not used because this basically disallows default case
'-Wunreachable-code-break',
'-Wunused-macros',
'-Wunused-parameter',
'-Wredundant-decls',
'-Wheader-guard',
'-Wno-format-nonliteral', # This is required because we pass format string as "const char*.
]),
language : 'c',
)
add_project_arguments('-D_GNU_SOURCE', language : 'c')
os_defines = []
if host_machine.system() == 'windows'
os_defines += '-DBZ_LCCWIN32=1'
os_defines += '-DBZ_UNIX=0'
else
os_defines += '-DBZ_LCCWIN32=0'
os_defines += '-DBZ_UNIX=1'
endif
c_args = []
# The or is a workaround for https://github.com/mesonbuild/meson/issues/5530
if cc.has_function_attribute('visibility') or (cc.get_id() == 'clang' and host_machine.system() == 'darwin')
c_args += '-DBZ_EXTERN=__attribute__((__visibility__("default")))'
endif
bz_sources = ['blocksort.c', 'huffman.c', 'crctable.c', 'randtable.c', 'compress.c', 'decompress.c', 'bzlib.c']
## Library versioning
##
## New package version:
## revision += 1
##
## New interfaces:
## current += 1
## revision = 0
## age += 1
##
## Deleted/changed interfaces:
## current += 1
## revision = 0
## age = 0
##
## KEEP THESE IN SYNC WITH CMakeLists.txt OR STUFF WILL BREAK!
bz2_lt_current = 1
bz2_lt_revision = 9
bz2_lt_age = 0
bz2_soversion = bz2_lt_current - bz2_lt_age
bz2_lt_version = '@0@.@1@.@2@'.format(bz2_soversion, bz2_lt_age, bz2_lt_revision)
if ['msvc', 'clang-cl', 'intel-cl'].contains(cc.get_id())
libbzip2 = library(
'bz2',
bz_sources,
c_args : c_args,
vs_module_defs : 'libbz2.def',
version : bz2_lt_version,
soversion : bz2_soversion,
install : true,
)
else
libbzip2 = library(
'bz2',
bz_sources,
c_args : c_args,
gnu_symbol_visibility : 'hidden',
version : bz2_lt_version,
soversion : bz2_soversion,
install : true,
)
endif
bzip2 = executable(
'bzip2',
['bzip2.c'],
link_with : [libbzip2],
install : true,
c_args : os_defines,
)
executable(
'bzip2recover',
['bzip2recover.c'],
link_with : [libbzip2],
install : true,
c_args : os_defines,
)
## Install wrapper scripts
install_data(
'bzgrep', 'bzmore', 'bzdiff',
install_dir : get_option('bindir'),
install_mode : 'rwxr-xr-x',
)
## Create aliases. Use links if possible, but copies if not.
# Copies are mainly meant for windows, which doesn't have symlinks.
bindir = get_option('bindir')
targets = [['bzmore', 'bzless'], ['bzdiff', 'bzcmp'], ['bzgrep', 'bzegrep', 'bzfgrep'],
['bzip2', 'bunzip2', 'bzcat']]
extra_args = []
if host_machine.system() != 'windows' and build_machine.system() != 'windows'
extra_args = '--use-links'
endif
foreach t : targets
meson.add_install_script('install_links.py', get_option('bindir'), t, extra_args)
endforeach
## Generate pkg-config automaically from built library information
pkg = import('pkgconfig')
pkg.generate(
libbzip2,
description : 'Lossless, block-sorting data compression',
)
## install headers
install_headers('bzlib.h')
subdir('man')
subdir('docs')
subdir('tests')

5
thirdparty/bzip2/meson_options.txt vendored
View file

@ -1,5 +0,0 @@
option(
'docs',
type : 'feature',
description : 'generate documentation in html, pdf, and ps format',
)

31
thirdparty/bzip2/mk251.c vendored
View file

@ -1,31 +0,0 @@
/* Spew out a long sequence of the byte 251. When fed to bzip2
versions 1.0.0 or 1.0.1, causes it to die with internal error
1007 in blocksort.c. This assertion misses an extremely rare
case, which is fixed in this version (1.0.2) and above.
*/
/* ------------------------------------------------------------------
This file is part of bzip2/libbzip2, a program and library for
lossless, block-sorting data compression.
bzip2/libbzip2 version 1.1.0 of 6 September 2010
Copyright (C) 1996-2010 Julian Seward <jseward@acm.org>
Please read the WARNING, DISCLAIMER and PATENTS sections in the
README file.
This program is released under the terms of the license contained
in the file LICENSE.
------------------------------------------------------------------ */
#include <stdio.h>
int main ()
{
int i;
for (i = 0; i < 48500000 ; i++)
putchar(251);
return 0;
}

7
thirdparty/bzip2/patches-not-included.txt vendored
View file

@ -1,7 +0,0 @@
These are the patches from openSUSE that are not included in this Git
repository:
* bzip2-point-to-doc-pkg.patch - modifies the README file to mention a
bzip2-doc package instead of the bzip2.txt file from the source
distribution. This is suitable for distributions which repackage
the original files into different packages.

84
thirdparty/bzip2/randtable.c vendored
View file

@ -1,84 +0,0 @@
/*-------------------------------------------------------------*/
/*--- Table for randomising repetitive blocks ---*/
/*--- randtable.c ---*/
/*-------------------------------------------------------------*/
/* ------------------------------------------------------------------
This file is part of bzip2/libbzip2, a program and library for
lossless, block-sorting data compression.
bzip2/libbzip2 version 1.1.0 of 6 September 2010
Copyright (C) 1996-2010 Julian Seward <jseward@acm.org>
Please read the WARNING, DISCLAIMER and PATENTS sections in the
README file.
This program is released under the terms of the license contained
in the file LICENSE.
------------------------------------------------------------------ */
#include "bzlib_private.h"
/*---------------------------------------------*/
Int32 BZ2_rNums[512] = {
619, 720, 127, 481, 931, 816, 813, 233, 566, 247,
985, 724, 205, 454, 863, 491, 741, 242, 949, 214,
733, 859, 335, 708, 621, 574, 73, 654, 730, 472,
419, 436, 278, 496, 867, 210, 399, 680, 480, 51,
878, 465, 811, 169, 869, 675, 611, 697, 867, 561,
862, 687, 507, 283, 482, 129, 807, 591, 733, 623,
150, 238, 59, 379, 684, 877, 625, 169, 643, 105,
170, 607, 520, 932, 727, 476, 693, 425, 174, 647,
73, 122, 335, 530, 442, 853, 695, 249, 445, 515,
909, 545, 703, 919, 874, 474, 882, 500, 594, 612,
641, 801, 220, 162, 819, 984, 589, 513, 495, 799,
161, 604, 958, 533, 221, 400, 386, 867, 600, 782,
382, 596, 414, 171, 516, 375, 682, 485, 911, 276,
98, 553, 163, 354, 666, 933, 424, 341, 533, 870,
227, 730, 475, 186, 263, 647, 537, 686, 600, 224,
469, 68, 770, 919, 190, 373, 294, 822, 808, 206,
184, 943, 795, 384, 383, 461, 404, 758, 839, 887,
715, 67, 618, 276, 204, 918, 873, 777, 604, 560,
951, 160, 578, 722, 79, 804, 96, 409, 713, 940,
652, 934, 970, 447, 318, 353, 859, 672, 112, 785,
645, 863, 803, 350, 139, 93, 354, 99, 820, 908,
609, 772, 154, 274, 580, 184, 79, 626, 630, 742,
653, 282, 762, 623, 680, 81, 927, 626, 789, 125,
411, 521, 938, 300, 821, 78, 343, 175, 128, 250,
170, 774, 972, 275, 999, 639, 495, 78, 352, 126,
857, 956, 358, 619, 580, 124, 737, 594, 701, 612,
669, 112, 134, 694, 363, 992, 809, 743, 168, 974,
944, 375, 748, 52, 600, 747, 642, 182, 862, 81,
344, 805, 988, 739, 511, 655, 814, 334, 249, 515,
897, 955, 664, 981, 649, 113, 974, 459, 893, 228,
433, 837, 553, 268, 926, 240, 102, 654, 459, 51,
686, 754, 806, 760, 493, 403, 415, 394, 687, 700,
946, 670, 656, 610, 738, 392, 760, 799, 887, 653,
978, 321, 576, 617, 626, 502, 894, 679, 243, 440,
680, 879, 194, 572, 640, 724, 926, 56, 204, 700,
707, 151, 457, 449, 797, 195, 791, 558, 945, 679,
297, 59, 87, 824, 713, 663, 412, 693, 342, 606,
134, 108, 571, 364, 631, 212, 174, 643, 304, 329,
343, 97, 430, 751, 497, 314, 983, 374, 822, 928,
140, 206, 73, 263, 980, 736, 876, 478, 430, 305,
170, 514, 364, 692, 829, 82, 855, 953, 676, 246,
369, 970, 294, 750, 807, 827, 150, 790, 288, 923,
804, 378, 215, 828, 592, 281, 565, 555, 710, 82,
896, 831, 547, 261, 524, 462, 293, 465, 502, 56,
661, 821, 976, 991, 658, 869, 905, 758, 745, 193,
768, 550, 608, 933, 378, 286, 215, 979, 792, 961,
61, 688, 793, 644, 986, 403, 106, 366, 905, 644,
372, 567, 466, 434, 645, 210, 389, 550, 919, 135,
780, 773, 635, 389, 707, 100, 626, 958, 165, 504,
920, 176, 193, 713, 857, 265, 203, 50, 668, 108,
645, 990, 626, 197, 510, 357, 358, 850, 858, 364,
936, 638
};
/*-------------------------------------------------------------*/
/*--- end randtable.c ---*/
/*-------------------------------------------------------------*/

54
thirdparty/bzip2/spewG.c vendored
View file

@ -1,54 +0,0 @@
/* spew out a thoroughly gigantic file designed so that bzip2
can compress it reasonably rapidly. This is to help test
support for large files (> 2GB) in a reasonable amount of time.
I suggest you use the undocumented --exponential option to
bzip2 when compressing the resulting file; this saves a bit of
time. Note: *don't* bother with --exponential when compressing
Real Files; it'll just waste a lot of CPU time :-)
(but is otherwise harmless).
*/
/* ------------------------------------------------------------------
This file is part of bzip2/libbzip2, a program and library for
lossless, block-sorting data compression.
bzip2/libbzip2 version 1.1.0 of 6 September 2010
Copyright (C) 1996-2010 Julian Seward <jseward@acm.org>
Please read the WARNING, DISCLAIMER and PATENTS sections in the
README file.
This program is released under the terms of the license contained
in the file LICENSE.
------------------------------------------------------------------ */
#define _FILE_OFFSET_BITS 64
#include <stdio.h>
#include <stdlib.h>
/* The number of megabytes of junk to spew out (roughly) */
#define MEGABYTES 5000
#define N_BUF 1000000
char buf[N_BUF];
int main ( int argc, char** argv )
{
int ii, kk, p;
srandom(1);
setbuffer ( stdout, buf, N_BUF );
for (kk = 0; kk < MEGABYTES * 515; kk+=3) {
p = 25+random()%50;
for (ii = 0; ii < p; ii++)
printf ( "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" );
for (ii = 0; ii < p-1; ii++)
printf ( "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb" );
for (ii = 0; ii < p+1; ii++)
printf ( "ccccccccccccccccccccccccccccccccccccc" );
}
fflush(stdout);
return 0;
}

75
thirdparty/bzip2/tests/CMakeLists.txt vendored
View file

@ -1,75 +0,0 @@
#
# Paths to pass to our tests via environment variables
#
if(WIN32)
file(TO_NATIVE_PATH ${CMAKE_SOURCE_DIR} SOURCE)
file(TO_NATIVE_PATH ${CMAKE_BINARY_DIR} BUILD)
file(TO_NATIVE_PATH ${CMAKE_CURRENT_BINARY_DIR} TMP)
if(ENABLE_STATIC_LIB)
file(TO_NATIVE_PATH $<TARGET_FILE:bz2_static> LIBBZ2)
else()
file(TO_NATIVE_PATH $<TARGET_FILE:bz2> LIBBZ2)
endif()
file(TO_NATIVE_PATH $<TARGET_FILE:bzip2> BZIP2)
else() # Not WIN32
if(ENABLE_SHARED_LIB)
set(LD_LIBRARY_PATH $<TARGET_FILE_DIR:bz2>:$ENV{LD_LIBRARY_PATH})
endif()
set(SOURCE ${CMAKE_SOURCE_DIR})
set(BUILD ${CMAKE_BINARY_DIR})
set(TMP ${CMAKE_CURRENT_BINARY_DIR})
if(ENABLE_STATIC_LIB)
set(LIBBZ2 $<TARGET_FILE:bz2_static>)
else()
set(LIBBZ2 $<TARGET_FILE:bz2>)
endif()
set(BZIP2 $<TARGET_FILE:bzip2>)
endif()
set(ENVIRONMENT
PYTHONTRACEMALLOC=1 VERSION=${PROJECT_VERSION}
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}
DYLD_LIBRARY_PATH=${LD_LIBRARY_PATH}
PATH_SOURCE=${SOURCE}
PATH_BUILD=${BUILD}
PATH_TMP=${TMP}
PATH_LIBBZ2=${LIBBZ2}
PATH_BZIP2=${BZIP2}
)
#
# The Tests
# ~~~~~~~~~
#
# Run all tests with: `ctest`
# or: `ctest -V` for verbose output
#
# Run a specific test with the `-R` option, like this:
# `ctest -V -R quick
#
add_test(NAME quick COMMAND ${PythonTest_COMMAND};quick_test.py
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR})
if(Valgrind_FOUND)
set_property(TEST quick PROPERTY ENVIRONMENT ${ENVIRONMENT} VALGRIND=${Valgrind_EXECUTABLE})
else()
set_property(TEST quick PROPERTY ENVIRONMENT ${ENVIRONMENT})
endif()
if(EXISTS ${CMAKE_CURRENT_SOURCE_DIR}/input/bzip2-testfiles/README.md)
add_test(NAME large COMMAND ${PythonTest_COMMAND};large_test.py
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR})
if(Valgrind_FOUND)
set_property(TEST large PROPERTY ENVIRONMENT ${ENVIRONMENT} VALGRIND=${Valgrind_EXECUTABLE})
else()
set_property(TEST large PROPERTY ENVIRONMENT ${ENVIRONMENT})
endif()
endif()

69
thirdparty/bzip2/tests/README.md vendored
View file

@ -1,69 +0,0 @@
# Tests
BZip2 has two test suites:
1. The quick test suite is the original test suite. It is small and runs very
quickly to verify correct compression and decompression of simple files.
2. The large test suite is a large collection of test files gathered from
various sources. It includes not only good `.bz2` files but also bad ones.
The quick tests will run under Valgrind if Valgrind is installed on the system
and was discovered by CMake/Meson at build time. If you installed Valgrind after
build time, you may have to do a clean build for the Valgrind to be detected.
The slow tests have Valgrind disabled, because with it enabled it takes upwards
of 35 minutes to run.
## Running the Tests
Run the tests using CMake or Meson's test commands.
For CMake:
```sh
ctest -V
```
For Meson:
```sh
meson test -C builddir --print-errorlogs
```
## Quick Test Suite
The quick test suite is a small set of `.bz2` compressed files and original
reference files.
BZip2 must be able to:
1. Compress the reference files without error and decompress the newly created
compressed version into a file that matches the original reference file.
Multiple compression modes are tested.
2. Decompress the `.bz2` files without error. The decompressed file must match
the original reference file.
## Large Test Suite
The large test suite tests a collection of "interesting" `.bz2` files that can
be used to test bzip2 works correctly. They come from different projects.
The test files for the Large Test Suite are in a separate repository that was
added here as a Git submodule: https://gitlab.com/bzip2/bzip2-testfiles
To run the larger test suite, you must first pull down the submodule before
running `ctest`.
E.g.:
```sh
git submodule update --init --recursive
```
For each `.bz2` file found it is decompressed, recompressed and decompressed
again. Once with the default bzip2 settings and once in `--small` (`-s`) mode.
Each time after decompression, the resulting file is checked against the MD5
reference hash to verify that decompression worked correctly.
For each `.bz2.bad` file, decompression is also tried twice, first in default-
mode and again in small-mode. The bzip2 binary is expected to return either `1`
or `2` as exit status. Any other exit code is interpreted as failure.

BIN
thirdparty/bzip2/tests/input/quick/sample1.bz2 (Stored with Git LFS) vendored
View file

Binary file not shown.

BIN
thirdparty/bzip2/tests/input/quick/sample1.ref vendored
View file

Binary file not shown.

BIN
thirdparty/bzip2/tests/input/quick/sample2.bz2 (Stored with Git LFS) vendored
View file

Binary file not shown.

BIN
thirdparty/bzip2/tests/input/quick/sample2.ref vendored
View file

Binary file not shown.

BIN
thirdparty/bzip2/tests/input/quick/sample3.bz2 (Stored with Git LFS) vendored
View file

Binary file not shown.

30007
thirdparty/bzip2/tests/input/quick/sample3.ref vendored
View file

File diff suppressed because it is too large Load diff

134
thirdparty/bzip2/tests/large_test.py vendored
View file

@ -1,134 +0,0 @@
#!/usr/bin/env python3
'''
Test correct decompression of good and bad .bz2 files found in:
https://gitlab.com/bzip2/bzip2-testfiles
'''
__copyright__ = 'Copyright (C) 2022 Micah Snyder'
from hashlib import md5
import os
from pathlib import Path
import testcase
TRY_VALGRIND = False # You can enable this if you want, but it will take a very
# long time to run, like a half hour or more.
path_source = Path(os.getenv('PATH_SOURCE'))
def generate_test_function(*args):
'''
Generate a test with the given args.
'''
def foo(self):
self.run_test(*args)
return foo
class TC(testcase.TestCase):
def run_test(self, sample: Path, extra_arg: str = ''):
'''
Verify correct behavior when bzip2 tries to decompress the given file.
'''
if sample.suffix == '.bad':
# Try to decompress, with the expectation that it will fail, gracefully:
# - First with default settings
# - Second time using `--small` mode
# Decompress. We can keep the OG file.
cmd = [str(TC.bzip2), '--decompress', '--keep', '--stdout', str(sample)]
(ec, out, err) = self.execute(cmd, try_valgrind=TRY_VALGRIND)
# Check that bzip2 failed gracefully.
assert ec == 2 or ec == 1
# Decompress small-mode. We can keep the OG file.
cmd = [str(TC.bzip2), '--decompress', '--small', '--keep', '--stdout', str(sample)]
(ec, out, err) = self.execute(cmd, try_valgrind=TRY_VALGRIND)
# Check that bzip2 failed gracefully.
assert ec == 2 or ec == 1
elif sample.suffix == '.bz2':
# Try to decompress, with the expectation that it will succeed and
# that the decompressed file's MD5 matches the reference file.
#
# Then, compress the result again, and decompress it again, verifying
# once more that the (second) decompressed file's MD5 matches the reference file.
# Verify that the decompressed MD5 reference file DOES exist.
with (sample.parent / (sample.stem + '.md5')).open('r') as md5_file:
# Get the reference hash
ref_hash = md5_file.read().split(' ')[0].strip()
# Decompress. We can keep the OG file.
cmd = [str(TC.bzip2), '--decompress', '--keep', '--stdout', str(sample)]
if extra_arg != '':
cmd.append(extra_arg)
(ec, out, err) = self.execute(cmd, try_valgrind=TRY_VALGRIND)
# Check that bzip2 thinks it succeeded.
assert ec == 0
# Verify that the decompressed file matches the .md5 reference.
out_hash = md5(out).hexdigest()
assert out_hash == ref_hash
# Write it to a temp file
tempfile_path = TC.path_tmp / (sample.name + '.decompressed')
print(f'Writing decompressed {sample.name} file to disk as {tempfile_path.name}...')
with tempfile_path.open('wb') as tmpfile:
tmpfile.write(out)
# Compress. No need to keep the temp file.
cmd = [str(TC.bzip2), '--compress', '--stdout', str(tempfile_path)]
if extra_arg != '':
cmd.append(extra_arg)
(ec, out, err) = self.execute(cmd, try_valgrind=TRY_VALGRIND)
# Check that bzip2 thinks it succeeded.
assert ec == 0
# Write it to a temp file
tempfile_path = TC.path_tmp / (tempfile_path.name + '.bz2')
print(f'Writing compressed {sample.name} file to disk as {tempfile_path.name}...')
with tempfile_path.open('wb') as tmpfile:
tmpfile.write(out)
# Decompress the compressed tempfile. No point keeping it.
cmd = [str(TC.bzip2), '--decompress', '--stdout', str(tempfile_path)]
if extra_arg != '':
cmd.append(extra_arg)
(ec, out, err) = self.execute(cmd, try_valgrind=TRY_VALGRIND)
# Check that bzip2 thinks it succeeded.
assert ec == 0
# Calculate hash of decompressed file.
out_hash = md5(out).hexdigest()
# Verify the decomp-comp-decomped file still matches the ref hash.
assert out_hash == ref_hash
# loop through directories in 'bzip2/tests/input/bzip2-testfiles'...
#
# For each of those directories, run a test on the files within that have
# the '.bad' or '.bz2' suffix.
testfiles_path = path_source / 'tests' / 'input' / 'bzip2-testfiles'
if testfiles_path.is_dir:
for sample in testfiles_path.glob('**/*'):
if sample.suffix == '.bad' or sample.suffix == '.bz2':
# Generate a test function for the sample.
bug_validation_test = generate_test_function(sample)
# Add the test function to the test class.
setattr(TC, f'test_{sample.parent.name}_{sample.name}', bug_validation_test)
# Generate a test function for the sample.
bug_validation_test = generate_test_function(sample, '--small')
# Add the test function to the test class.
setattr(TC, f'test_{sample.parent.name}_{sample.name}_small', bug_validation_test)

46
thirdparty/bzip2/tests/meson.build vendored
View file

@ -1,46 +0,0 @@
prog_python = import('python').find_installation('python3')
r = run_command(prog_python, '-m', 'pytest', '--help', check: false)
errortxt = r.stderr().strip()
if errortxt.contains('No module named pytest')
error('pytest for python3 is required to run the tests. Please `pip3 install pytest` and try again.')
endif
valgrind = find_program('valgrind', required : false)
fs = import('fs')
env = environment()
environment = [
'PYTHONTRACEMALLOC=1',
'VERSION=' + meson.project_version(),
'LD_LIBRARY_PATH=' + fs.parent(libbzip2.full_path()),
'DYLD_LIBRARY_PATH=' + fs.parent(libbzip2.full_path()),
'PATH_SOURCE=' + meson.project_source_root(),
'PATH_BUILD=' + meson.project_build_root(),
'PATH_TMP=' + meson.current_build_dir(),
'PATH_LIBBZ2=' + libbzip2.full_path(),
'PATH_BZIP2=' + bzip2.full_path(),
]
if valgrind.found() == true
environment += 'VALGRIND=' + valgrind.full_path()
endif
test(
'quick_test.py',
prog_python,
args : ['-m', 'pytest', '-vv', meson.current_source_dir() + '/quick_test.py'],
env : environment,
timeout : 500
)
if (fs.exists(meson.current_source_dir() + '/input/bzip2-testfiles/README.md'))
test(
'large_test.py',
prog_python,
args : ['-m', 'pytest', '-vv', meson.current_source_dir() + '/large_test.py'],
env : environment,
timeout : 10000
)
endif

131
thirdparty/bzip2/tests/quick_test.py vendored
View file

@ -1,131 +0,0 @@
#!/usr/bin/env python3
'''
basic compression/decompression tests.
'''
__copyright__ = 'Copyright (C) 2022 Micah Snyder'
from hashlib import md5
import os
from pathlib import Path
import testcase
TRY_VALGRIND = True
path_source = Path(os.getenv('PATH_SOURCE'))
def generate_test_function(*args):
'''
Generate a test with the given args.
'''
def foo(self):
self.run_test(*args)
return foo
class TC(testcase.TestCase):
def tearDown(self):
super().tearDown()
self.verify_valgrind_log()
def run_test(self, sample: Path, block_size: int = 0):
'''
Verify correct behavior when bzip2 tries to decompress the given file.
Note: block_size is only used for the compression test.
'''
if sample.suffix == '.ref':
# Try to decompress, with the expectation that it will fail, gracefully:
# - First with default settings
# - Second time using `--small` mode
# Compress. We can keep the OG file.
cmd = [str(TC.bzip2), '--compress', str(block_size), '--keep', '--stdout', str(sample)]
(ec, out, err) = self.execute(cmd, try_valgrind=TRY_VALGRIND)
# Check that bzip2 thinks it succeeded.
assert ec == 0
# Write it to a temp file
tempfile_path = TC.path_tmp / (sample.name + '.bz2')
print(f'Writing compressed {sample.name} file to disk as {tempfile_path.name}...')
with tempfile_path.open('wb') as tmpfile:
tmpfile.write(out)
# Decompress the compresed tempfile. No point keeping it.
cmd = [str(TC.bzip2), '--decompress', '--stdout', str(tempfile_path)]
(ec, out, err) = self.execute(cmd, try_valgrind=TRY_VALGRIND)
# Check that bzip2 thinks it succeeded.
assert ec == 0
# Calculate hash of decompressed file.
out_hash = md5(out).hexdigest()
# Verify that the MD5 of the original reference file matches MD5 of
# the compressed & decompressed file.
with sample.open('rb') as reffile:
# Calcualte hash of reference file.
refcontents = reffile.read()
ref_hash = md5(refcontents).hexdigest()
print(f'Checking that {tempfile_path.name} matches {sample.name} when decompressed...')
assert out_hash == ref_hash, \
'decompression output and reference file differ:\n' + \
TC.hex_compare(out, refcontents)
elif sample.suffix == '.bz2':
# Try to decompress, with the expectation that it will succeed and
# that the decompressed file's MD5 matches the reference file.
#
# Then, compress the result again, and decompress it again, verifying
# once more that the (second) decompressed file's MD5 matches the reference file.
# Verify that the MD5 of the original reference file matches MD5 of
# the decompressed file.
reffile_path = sample.parent / (sample.stem + '.ref')
with reffile_path.open('rb') as reffile:
# Calcualte hash of reference file.
refcontents = reffile.read()
ref_hash = md5(refcontents).hexdigest()
# Decompress. We can keep the OG file.
cmd = [str(TC.bzip2), '--decompress', '--keep', '--stdout', str(sample)]
(ec, out, err) = self.execute(cmd, try_valgrind=TRY_VALGRIND)
# Check that bzip2 thinks it succeeded.
assert ec == 0
# Calculate hash of decompressed file.
out_hash = md5(out).hexdigest()
# Verify that the decompressed file matches the MD5 of the original reference file.
print(f'Checking that {sample.name} matches {reffile_path.name} when decompressed...')
assert out_hash == ref_hash, \
'decompression output and reference file differ:\n' + \
TC.hex_compare(out, refcontents)
# loop through directories in 'bzip2/tests/input/quick'...
#
# For each of those directories, run a test on the files within that have
# the '.bad' or '.bz2' suffix.
testfiles_path = path_source / 'tests' / 'input' / 'quick'
if testfiles_path.is_dir:
for sample in testfiles_path.glob('**/*'):
if sample.suffix == '.ref':
for block_size in [-1, -2, -3]:
# Generate a test function for the sample.
bug_validation_test = generate_test_function(sample, block_size)
# Add the test function to the test class.
setattr(TC, f'test_comp_decomp_{sample.name}_{abs(block_size)}', bug_validation_test)
elif sample.suffix == '.bz2':
# Generate a test function for the sample.
bug_validation_test = generate_test_function(sample)
# Add the test function to the test class.
setattr(TC, f'test_decomp_only_{sample.name}', bug_validation_test)

215
thirdparty/bzip2/tests/testcase.py vendored
View file

@ -1,215 +0,0 @@
'''
Wrapper for Python's unittest.TestCase that sets up BZip2 testing environment.
'''
__copyright__ = "Copyright (C) 2022 Micah Snyder"
from math import ceil
import os
from pathlib import Path
import platform
import shutil
import subprocess
import tempfile
import unittest
from typing import Tuple, Union, NamedTuple
# Use older Python 3.5 syntax.
CmdResult = NamedTuple('CmdResult', [('ec', int), ('out', bytes), ('err', bytes)])
class TestCase(unittest.TestCase):
version = ""
path_source = None
path_build = None
path_tmp = None
bzip2 = None
valgrind = "" # Not 'None' because we'll use this variable even if valgrind not found.
valgrind_args = []
original_working_directory = ""
@classmethod
def setUpClass(cls):
'''
Prepare test environment:
- Create a temporary testing directory.
- Get paths needed for tests from environment variables.
'''
cls.operating_system = platform.platform().split("-")[0].lower()
# The bzip2 program uses the BZIP and BZIP2 environment variables as
# additional input. We must purge them to prevent OS environment
# variables from affecting the test suite.
os.environ.pop('BZIP', None)
os.environ.pop('BZIP2', None)
# Version may be used for testing bzip2 --version output, etc.
cls.version = os.getenv("VERSION")
if cls.version == None:
raise Exception("VERSION environment variable not defined! Aborting...")
# Get test paths from environment variables.
cls.path_source = Path(os.getenv("PATH_SOURCE"))
cls.path_build = Path(os.getenv("PATH_BUILD"))
cls.bzip2 = Path(os.getenv("PATH_BZIP2")) if os.getenv("PATH_BZIP2") != None else None
# Generate temp directory
cls.path_tmp = Path(tempfile.mkdtemp(prefix=(cls.__name__ + "-"), dir=os.getenv("TMP")))
# Enable valgrind testing if VALGRIND variable set to path of Valgrind executable.
if os.getenv('VALGRIND') != None:
valgrind = Path(os.getenv("VALGRIND"))
if valgrind.is_file():
cls.valgrind = valgrind
logfile = cls.path_tmp / 'valgrind.log'
cls.valgrind_args = [
'-v',
'--trace-children=yes',
'--track-fds=yes',
'--leak-check=full',
'--gen-suppressions=all',
'--show-leak-kinds=definite',
'--errors-for-leak-kinds=definite',
f'--log-file={logfile}',
'--error-exitcode=123',
]
# Perform all tests with cwd set to the cls.path_tmp, created above.
cls.original_working_directory = os.getcwd()
os.chdir(cls.path_tmp)
@classmethod
def tearDownClass(cls):
'''
Clean up after ourselves,
Delete the generated tmp directory.
'''
print("")
# Restore current working directory before deleting cls.path_tmp.
os.chdir(cls.original_working_directory)
if None == os.getenv("KEEPTEMP"):
try:
shutil.rmtree(cls.path_tmp)
print("Removed tmp directory: {}".format(cls.path_tmp))
except Exception:
print("No tmp directory to clean up.")
def setUp(self):
print('\n')
def tearDown(self):
print('\n')
def execute(self, cmd: list, try_valgrind: bool = True) -> CmdResult:
'''
Execute a subprocess.Popen list of commands.
Return a tuple of
'''
# Use valgrind if we have it.
if try_valgrind and self.valgrind != '':
cmd = [str(self.valgrind),] + self.valgrind_args + cmd
print(f"Running: {' '.join(cmd)}\n")
p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = p.communicate()
# Check the valgrind log for errors.
if try_valgrind and self.valgrind != '':
self.verify_valgrind_log()
return CmdResult(p.returncode, out, err)
def verify_valgrind_log(self, log_file: Union[Path, None]=None):
'''
Check if valgrind log file contains errors.
If valgrind not enabled this is basically a nop.
'''
if self.valgrind == "":
return
if log_file == None:
log_file = self.path_tmp / 'valgrind.log'
if not log_file.exists():
raise AssertionError('{} not found. Valgrind failed to run?'.format(log_file))
errors = False
print('Verifying {}...'.format(log_file))
try:
with log_file.open('r') as the_log:
assert 'ERROR SUMMARY: 0 errors' not in the_log
except AssertionError:
print("*" * 80)
print('Valgrind test failed!'.center(80, ' '))
print('Please submit a bug report with this log to https://gitlab.com/bzip2/bzip2/issues'.center(69, ' '))
print(str(log_file).center(80, ' '))
print("*" * 80)
errors = True
finally:
with log_file.open('r') as log:
found_summary = False
for line in log.readlines():
if 'ERROR SUMMARY' in line:
found_summary = True
if (found_summary or errors) and len(line) < 500:
print(line.rstrip('\n'))
if errors:
raise AssertionError('Valgrind test FAILED!')
@staticmethod
def hex_compare(actual: bytes, expected: bytes, size: int = 16):
'''
Return string with hex comparison of two buffers
'''
a_lines = ceil(float(len(actual)) / float(size))
e_lines = ceil(float(len(expected)) / float(size))
lines = max(a_lines, e_lines)
comparison = ' ' + \
f'output ({len(actual)}):'.ljust(size*2 + 3) + \
f'expected ({len(expected)}):\n'
def render_slice(to_print, to_compare):
line = ''
for byte in range(0, size):
if byte == size / 2:
line += ' '
if byte < len(to_print):
if byte >= len(to_compare) or to_print[byte] != to_compare[byte]:
line += '\x1b[1;33m{:02x}\x1b[0m'.format(to_print[byte]) # bold yellow
else:
line += '{:02x}'.format(to_print[byte]) # plain
else:
line += ' '
return line
prev_is_dots = False
for line in range(0, lines):
a_line = actual[line * size : line * size + size]
e_line = expected[line * size : line * size + size]
if a_line == e_line:
if prev_is_dots == False:
comparison += " ...\n"
prev_is_dots = True
else:
text_line = '{:8d}: {} {}'.format(
line * size,
render_slice(a_line, e_line),
render_slice(e_line, a_line)
)
comparison += text_line + '\n'
prev_is_dots = False
return comparison + '\n'

141
thirdparty/bzip2/unzcrash.c vendored
View file

@ -1,141 +0,0 @@
/* A test program written to test robustness to decompression of
corrupted data. Usage is
unzcrash filename
and the program will read the specified file, compress it (in memory),
and then repeatedly decompress it, each time with a different bit of
the compressed data inverted, so as to test all possible one-bit errors.
This should not cause any invalid memory accesses. If it does,
I want to know about it!
PS. As you can see from the above description, the process is
incredibly slow. A file of size eg 5KB will cause it to run for
many hours.
*/
/* ------------------------------------------------------------------
This file is part of bzip2/libbzip2, a program and library for
lossless, block-sorting data compression.
bzip2/libbzip2 version 1.1.0 of 6 September 2010
Copyright (C) 1996-2010 Julian Seward <jseward@acm.org>
Please read the WARNING, DISCLAIMER and PATENTS sections in the
README file.
This program is released under the terms of the license contained
in the file LICENSE.
------------------------------------------------------------------ */
#include <stdio.h>
#include <assert.h>
#include "bzlib.h"
#define M_BLOCK 1000000
typedef unsigned char uchar;
#define M_BLOCK_OUT (M_BLOCK + 1000000)
uchar inbuf[M_BLOCK];
uchar outbuf[M_BLOCK_OUT];
uchar zbuf[M_BLOCK + 600 + (M_BLOCK / 100)];
int nIn, nOut, nZ;
static char *bzerrorstrings[] = {
"OK"
,"SEQUENCE_ERROR"
,"PARAM_ERROR"
,"MEM_ERROR"
,"DATA_ERROR"
,"DATA_ERROR_MAGIC"
,"IO_ERROR"
,"UNEXPECTED_EOF"
,"OUTBUFF_FULL"
,"???" /* for future */
,"???" /* for future */
,"???" /* for future */
,"???" /* for future */
,"???" /* for future */
,"???" /* for future */
};
void flip_bit ( int bit )
{
int byteno = bit / 8;
int bitno = bit % 8;
uchar mask = 1 << bitno;
//fprintf ( stderr, "(byte %d bit %d mask %d)",
// byteno, bitno, (int)mask );
zbuf[byteno] ^= mask;
}
int main ( int argc, char** argv )
{
FILE* f;
int r;
int bit;
int i;
if (argc != 2) {
fprintf ( stderr, "usage: unzcrash filename\n" );
return 1;
}
f = fopen ( argv[1], "r" );
if (!f) {
fprintf ( stderr, "unzcrash: can't open %s\n", argv[1] );
return 1;
}
nIn = fread ( inbuf, 1, M_BLOCK, f );
fprintf ( stderr, "%d bytes read\n", nIn );
nZ = M_BLOCK;
r = BZ2_bzBuffToBuffCompress (
zbuf, &nZ, inbuf, nIn, 9, 0, 30 );
assert (r == BZ_OK);
fprintf ( stderr, "%d after compression\n", nZ );
for (bit = 0; bit < nZ*8; bit++) {
fprintf ( stderr, "bit %d ", bit );
flip_bit ( bit );
nOut = M_BLOCK_OUT;
r = BZ2_bzBuffToBuffDecompress (
outbuf, &nOut, zbuf, nZ, 0, 0 );
fprintf ( stderr, " %d %s ", r, bzerrorstrings[-r] );
if (r != BZ_OK) {
fprintf ( stderr, "\n" );
} else {
if (nOut != nIn) {
fprintf(stderr, "nIn/nOut mismatch %d %d\n", nIn, nOut );
return 1;
} else {
for (i = 0; i < nOut; i++)
if (inbuf[i] != outbuf[i]) {
fprintf(stderr, "mismatch at %d\n", i );
return 1;
}
if (i == nOut) fprintf(stderr, "really ok!\n" );
}
}
flip_bit ( bit );
}
#if 0
assert (nOut == nIn);
for (i = 0; i < nOut; i++) {
if (inbuf[i] != outbuf[i]) {
fprintf ( stderr, "difference at %d !\n", i );
return 1;
}
}
#endif
fprintf ( stderr, "all ok\n" );
return 0;
}

48
thirdparty/bzip2/version.rc vendored
View file

@ -1,48 +0,0 @@
#include <winver.h>
LANGUAGE 0x09,0x01
1 VERSIONINFO
FILEVERSION 1,0,7,0
PRODUCTVERSION 1,0,7,0
FILEFLAGSMASK 0x3fL
FILEFLAGS 0x00L
FILEOS VOS__WINDOWS32
#if defined(BZ21DLL)
FILETYPE VFT_DLL
#else
FILETYPE VFT_APP
#endif
FILESUBTYPE 0x0L
BEGIN
BLOCK "StringFileInfo"
BEGIN
BLOCK "040904b0"
BEGIN
// VALUE "Comments", "\0"
VALUE "CompanyName", "bzip2, https://gitlab.com/bzip2/bzip2\0"
VALUE "FileDescription", "bzip2\0"
VALUE "FileVersion", "1.1.0\0"
#if defined(BZ21DLL)
VALUE "InternalName", "bz2-1\0"
VALUE "OriginalFilename", "bz2-1.dll\0"
#elif defined(BZIP2)
VALUE "InternalName", "bzip2\0"
VALUE "OriginalFilename", "bzip2.exe\0"
#elif defined(BZIP2RECOVER)
VALUE "InternalName", "bzip2recover\0"
VALUE "OriginalFilename", "bzip2recover.exe\0"
#endif
VALUE "LegalCopyright", "Copyright (C) 1996-2010 Julian Seward <jseward@acm.org>. Copyright (C) 2019 Federico Mena Quintero <federico@gnome.org>. All rights reserved.\0"
// VALUE "LegalTrademarks", "\0"
// VALUE "PrivateBuild", "\0"
VALUE "ProductName", "bzip2\0"
VALUE "ProductVersion", "1.1.0\0"
// VALUE "SpecialBuild", "\0"
END
END
BLOCK "VarFileInfo"
BEGIN
VALUE "Translation", 0x409, 0x4b0
END
END

54
thirdparty/bzip2/version.rc.in vendored
View file

@ -1,54 +0,0 @@
#include <winver.h>
VS_VERSION_INFO VERSIONINFO
FILEVERSION @PROJECT_VERSION_MAJOR@, @PROJECT_VERSION_MINOR@, @PROJECT_VERSION_PATCH@, 0
PRODUCTVERSION @PROJECT_VERSION_MAJOR@, @PROJECT_VERSION_MINOR@, @PROJECT_VERSION_PATCH@, 0
FILEFLAGSMASK 0x3fL
FILEOS VOS__WINDOWS32
#if defined(LIBBZ2)
FILETYPE VFT_DLL
#else
FILETYPE VFT_APP
#endif
FILESUBTYPE 0x0L
#ifdef _DEBUG
#define VER_STR "@PROJECT_VERSION@.0 (MSVC debug)"
#define DBG "d"
FILEFLAGS 0x1L
#else
#define VER_STR "@PROJECT_VERSION@.0 (MSVC release)"
#define DBG ""
FILEFLAGS 0x0L
#endif
BEGIN
BLOCK "StringFileInfo"
BEGIN
BLOCK "040904b0"
BEGIN
// VALUE "Comments", ""
VALUE "CompanyName", "bzip2, https://gitlab.com/bzip2/bzip2"
VALUE "FileDescription", "bzip2"
VALUE "FileVersion", "1.0.6"
#if defined(LIBBZ2)
VALUE "InternalName", "libbz2"
VALUE "OriginalFilename", "libbz2.dll"
#elif defined(BZIP2)
VALUE "InternalName", "bzip2"
VALUE "OriginalFilename", "bzip2.exe"
#elif defined(BZIP2RECOVER)
VALUE "InternalName", "bzip2recover"
VALUE "OriginalFilename", "bzip2recover.exe"
#endif
VALUE "LegalCopyright", "Copyright (C) 1996-2010 Julian Seward <jseward@acm.org>. All rights reserved."
VALUE "LegalTrademarks", ""
VALUE "ProductName", "bzip2"
VALUE "ProductVersion", "1.0.6"
// VALUE "SpecialBuild", ""
END
END
BLOCK "VarFileInfo"
BEGIN
VALUE "Translation", 0x409, 0x4b0
END
END

18
thirdparty/ffms2/.gitattributes vendored
View file

@ -1,18 +0,0 @@
# Auto detect text files and perform LF normalization
* text=auto
# But don't mess up Unix scripts in the process
*.sh eol=lf
*.in eol=lf
*.am eol=lf
*.ac eol=lf
*.m4 eol=lf
# Scripts that don't have extensions...
/install-sh eol=lf
/compile eol=lf
/configure eol=lf
/config.guess eol=lf
/config.sub eol=lf
/depcomp eol=lf
/install-sh eol=lf
/missing eol=lf

93
thirdparty/ffms2/.gitignore vendored
View file

@ -1,93 +0,0 @@
# /
/deps
/testing
/todo.txt
# /build-msvc/
/build-msvc/*.sdf
/build-msvc/*.suo
/build-msvc/*.user
/build-msvc/ffms2.ncb
/build-msvc/ffmsindex
/build-msvc/bin
/build-msvc/obj
# /src/config/
/src/config/config.h
/src/config/config.h.in~
/src/config/auto_config.h
/src/config/stamp-h1
# /src/core/
/src/core/.deps
/src/core/.libs
/src/core/.dirstamp
/src/core/*.o
/src/core/*.lo
/src/core/*.la
# /src/index/
/src/index/.deps
/src/index/.libs
/src/index/.dirstamp
/src/index/*.o
/src/index/*.exe
/src/index/ffmsindex
# /src/vapoursynth/
/src/vapoursynth/.deps
/src/vapoursynth/.libs
/src/vapoursynth/.dirstamp
/src/vapoursynth/*.o
/src/vapoursynth/*.lo
# /test
/test/samples
/test/display_matrix
/test/hdr
/test/.libs
/test/*.o
/test/*.a
/test/indexer
### Autotools ###
# automake
Makefile.in
.deps/
# autoconf
/ffms2.pc
/autom4te.cache
/aclocal.m4
/compile
/config.cache
/config.guess
/config.guess~
config.h.in
/config.log
/config.status
/config.sub
/config.sub~
/configure
/configure~
/configure.scan
/depcomp
/install-sh
/missing
/stamp-h1
# libtool
/libtool
/ltmain.sh
# m4
m4/libtool.m4
m4/ltoptions.m4
m4/ltsugar.m4
m4/ltversion.m4
m4/lt~obsolete.m4
# Generated Makefile
Makefile

31
thirdparty/ffms2/COPYING vendored
View file

@ -1,31 +0,0 @@
The FFMS2 source is licensed under the MIT license, but its binaries
are licensed under the GPL because GPL components of FFmpeg are
used. FFmpeg can be built as either LGPL, GPLv2, GPLv3, or even be
nonredistributable. Refer to FFmpeg's sources for licensing information.
Text of MIT license:
-----------------------------------------------------------
Copyright (c) <year> <copyright holders>
Permission is hereby granted, free of charge, to any person
obtaining a copy of this software and associated documentation
files (the "Software"), to deal in the Software without
restriction, including without limitation the rights to use,
copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the
Software is furnished to do so, subject to the following
conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
OTHER DEALINGS IN THE SOFTWARE.
-----------------------------------------------------------

365
thirdparty/ffms2/INSTALL vendored
View file

@ -1,365 +0,0 @@
Installation Instructions
*************************
Copyright (C) 1994, 1995, 1996, 1999, 2000, 2001, 2002, 2004, 2005,
2006, 2007, 2008, 2009 Free Software Foundation, Inc.
Copying and distribution of this file, with or without modification,
are permitted in any medium without royalty provided the copyright
notice and this notice are preserved. This file is offered as-is,
without warranty of any kind.
Basic Installation
==================
Briefly, the shell commands `./configure; make; make install' should
configure, build, and install this package. The following
more-detailed instructions are generic; see the `README' file for
instructions specific to this package. Some packages provide this
`INSTALL' file but do not implement all of the features documented
below. The lack of an optional feature in a given package is not
necessarily a bug. More recommendations for GNU packages can be found
in *note Makefile Conventions: (standards)Makefile Conventions.
The `configure' shell script attempts to guess correct values for
various system-dependent variables used during compilation. It uses
those values to create a `Makefile' in each directory of the package.
It may also create one or more `.h' files containing system-dependent
definitions. Finally, it creates a shell script `config.status' that
you can run in the future to recreate the current configuration, and a
file `config.log' containing compiler output (useful mainly for
debugging `configure').
It can also use an optional file (typically called `config.cache'
and enabled with `--cache-file=config.cache' or simply `-C') that saves
the results of its tests to speed up reconfiguring. Caching is
disabled by default to prevent problems with accidental use of stale
cache files.
If you need to do unusual things to compile the package, please try
to figure out how `configure' could check whether to do them, and mail
diffs or instructions to the address given in the `README' so they can
be considered for the next release. If you are using the cache, and at
some point `config.cache' contains results you don't want to keep, you
may remove or edit it.
The file `configure.ac' (or `configure.in') is used to create
`configure' by a program called `autoconf'. You need `configure.ac' if
you want to change it or regenerate `configure' using a newer version
of `autoconf'.
The simplest way to compile this package is:
1. `cd' to the directory containing the package's source code and type
`./configure' to configure the package for your system.
Running `configure' might take a while. While running, it prints
some messages telling which features it is checking for.
2. Type `make' to compile the package.
3. Optionally, type `make check' to run any self-tests that come with
the package, generally using the just-built uninstalled binaries.
4. Type `make install' to install the programs and any data files and
documentation. When installing into a prefix owned by root, it is
recommended that the package be configured and built as a regular
user, and only the `make install' phase executed with root
privileges.
5. Optionally, type `make installcheck' to repeat any self-tests, but
this time using the binaries in their final installed location.
This target does not install anything. Running this target as a
regular user, particularly if the prior `make install' required
root privileges, verifies that the installation completed
correctly.
6. You can remove the program binaries and object files from the
source code directory by typing `make clean'. To also remove the
files that `configure' created (so you can compile the package for
a different kind of computer), type `make distclean'. There is
also a `make maintainer-clean' target, but that is intended mainly
for the package's developers. If you use it, you may have to get
all sorts of other programs in order to regenerate files that came
with the distribution.
7. Often, you can also type `make uninstall' to remove the installed
files again. In practice, not all packages have tested that
uninstallation works correctly, even though it is required by the
GNU Coding Standards.
8. Some packages, particularly those that use Automake, provide `make
distcheck', which can by used by developers to test that all other
targets like `make install' and `make uninstall' work correctly.
This target is generally not run by end users.
Compilers and Options
=====================
Some systems require unusual options for compilation or linking that
the `configure' script does not know about. Run `./configure --help'
for details on some of the pertinent environment variables.
You can give `configure' initial values for configuration parameters
by setting variables in the command line or in the environment. Here
is an example:
./configure CC=c99 CFLAGS=-g LIBS=-lposix
*Note Defining Variables::, for more details.
Compiling For Multiple Architectures
====================================
You can compile the package for more than one kind of computer at the
same time, by placing the object files for each architecture in their
own directory. To do this, you can use GNU `make'. `cd' to the
directory where you want the object files and executables to go and run
the `configure' script. `configure' automatically checks for the
source code in the directory that `configure' is in and in `..'. This
is known as a "VPATH" build.
With a non-GNU `make', it is safer to compile the package for one
architecture at a time in the source code directory. After you have
installed the package for one architecture, use `make distclean' before
reconfiguring for another architecture.
On MacOS X 10.5 and later systems, you can create libraries and
executables that work on multiple system types--known as "fat" or
"universal" binaries--by specifying multiple `-arch' options to the
compiler but only a single `-arch' option to the preprocessor. Like
this:
./configure CC="gcc -arch i386 -arch x86_64 -arch ppc -arch ppc64" \
CXX="g++ -arch i386 -arch x86_64 -arch ppc -arch ppc64" \
CPP="gcc -E" CXXCPP="g++ -E"
This is not guaranteed to produce working output in all cases, you
may have to build one architecture at a time and combine the results
using the `lipo' tool if you have problems.
Installation Names
==================
By default, `make install' installs the package's commands under
`/usr/local/bin', include files under `/usr/local/include', etc. You
can specify an installation prefix other than `/usr/local' by giving
`configure' the option `--prefix=PREFIX', where PREFIX must be an
absolute file name.
You can specify separate installation prefixes for
architecture-specific files and architecture-independent files. If you
pass the option `--exec-prefix=PREFIX' to `configure', the package uses
PREFIX as the prefix for installing programs and libraries.
Documentation and other data files still use the regular prefix.
In addition, if you use an unusual directory layout you can give
options like `--bindir=DIR' to specify different values for particular
kinds of files. Run `configure --help' for a list of the directories
you can set and what kinds of files go in them. In general, the
default for these options is expressed in terms of `${prefix}', so that
specifying just `--prefix' will affect all of the other directory
specifications that were not explicitly provided.
The most portable way to affect installation locations is to pass the
correct locations to `configure'; however, many packages provide one or
both of the following shortcuts of passing variable assignments to the
`make install' command line to change installation locations without
having to reconfigure or recompile.
The first method involves providing an override variable for each
affected directory. For example, `make install
prefix=/alternate/directory' will choose an alternate location for all
directory configuration variables that were expressed in terms of
`${prefix}'. Any directories that were specified during `configure',
but not in terms of `${prefix}', must each be overridden at install
time for the entire installation to be relocated. The approach of
makefile variable overrides for each directory variable is required by
the GNU Coding Standards, and ideally causes no recompilation.
However, some platforms have known limitations with the semantics of
shared libraries that end up requiring recompilation when using this
method, particularly noticeable in packages that use GNU Libtool.
The second method involves providing the `DESTDIR' variable. For
example, `make install DESTDIR=/alternate/directory' will prepend
`/alternate/directory' before all installation names. The approach of
`DESTDIR' overrides is not required by the GNU Coding Standards, and
does not work on platforms that have drive letters. On the other hand,
it does better at avoiding recompilation issues, and works well even
when some directory options were not specified in terms of `${prefix}'
at `configure' time.
Optional Features
=================
If the package supports it, you can cause programs to be installed
with an extra prefix or suffix on their names by giving `configure' the
option `--program-prefix=PREFIX' or `--program-suffix=SUFFIX'.
Some packages pay attention to `--enable-FEATURE' options to
`configure', where FEATURE indicates an optional part of the package.
They may also pay attention to `--with-PACKAGE' options, where PACKAGE
is something like `gnu-as' or `x' (for the X Window System). The
`README' should mention any `--enable-' and `--with-' options that the
package recognizes.
For packages that use the X Window System, `configure' can usually
find the X include and library files automatically, but if it doesn't,
you can use the `configure' options `--x-includes=DIR' and
`--x-libraries=DIR' to specify their locations.
Some packages offer the ability to configure how verbose the
execution of `make' will be. For these packages, running `./configure
--enable-silent-rules' sets the default to minimal output, which can be
overridden with `make V=1'; while running `./configure
--disable-silent-rules' sets the default to verbose, which can be
overridden with `make V=0'.
Particular systems
==================
On HP-UX, the default C compiler is not ANSI C compatible. If GNU
CC is not installed, it is recommended to use the following options in
order to use an ANSI C compiler:
./configure CC="cc -Ae -D_XOPEN_SOURCE=500"
and if that doesn't work, install pre-built binaries of GCC for HP-UX.
On OSF/1 a.k.a. Tru64, some versions of the default C compiler cannot
parse its `<wchar.h>' header file. The option `-nodtk' can be used as
a workaround. If GNU CC is not installed, it is therefore recommended
to try
./configure CC="cc"
and if that doesn't work, try
./configure CC="cc -nodtk"
On Solaris, don't put `/usr/ucb' early in your `PATH'. This
directory contains several dysfunctional programs; working variants of
these programs are available in `/usr/bin'. So, if you need `/usr/ucb'
in your `PATH', put it _after_ `/usr/bin'.
On Haiku, software installed for all users goes in `/boot/common',
not `/usr/local'. It is recommended to use the following options:
./configure --prefix=/boot/common
Specifying the System Type
==========================
There may be some features `configure' cannot figure out
automatically, but needs to determine by the type of machine the package
will run on. Usually, assuming the package is built to be run on the
_same_ architectures, `configure' can figure that out, but if it prints
a message saying it cannot guess the machine type, give it the
`--build=TYPE' option. TYPE can either be a short name for the system
type, such as `sun4', or a canonical name which has the form:
CPU-COMPANY-SYSTEM
where SYSTEM can have one of these forms:
OS
KERNEL-OS
See the file `config.sub' for the possible values of each field. If
`config.sub' isn't included in this package, then this package doesn't
need to know the machine type.
If you are _building_ compiler tools for cross-compiling, you should
use the option `--target=TYPE' to select the type of system they will
produce code for.
If you want to _use_ a cross compiler, that generates code for a
platform different from the build platform, you should specify the
"host" platform (i.e., that on which the generated programs will
eventually be run) with `--host=TYPE'.
Sharing Defaults
================
If you want to set default values for `configure' scripts to share,
you can create a site shell script called `config.site' that gives
default values for variables like `CC', `cache_file', and `prefix'.
`configure' looks for `PREFIX/share/config.site' if it exists, then
`PREFIX/etc/config.site' if it exists. Or, you can set the
`CONFIG_SITE' environment variable to the location of the site script.
A warning: not all `configure' scripts look for a site script.
Defining Variables
==================
Variables not defined in a site shell script can be set in the
environment passed to `configure'. However, some packages may run
configure again during the build, and the customized values of these
variables may be lost. In order to avoid this problem, you should set
them in the `configure' command line, using `VAR=value'. For example:
./configure CC=/usr/local2/bin/gcc
causes the specified `gcc' to be used as the C compiler (unless it is
overridden in the site shell script).
Unfortunately, this technique does not work for `CONFIG_SHELL' due to
an Autoconf bug. Until the bug is fixed you can use this workaround:
CONFIG_SHELL=/bin/bash /bin/bash ./configure CONFIG_SHELL=/bin/bash
`configure' Invocation
======================
`configure' recognizes the following options to control how it
operates.
`--help'
`-h'
Print a summary of all of the options to `configure', and exit.
`--help=short'
`--help=recursive'
Print a summary of the options unique to this package's
`configure', and exit. The `short' variant lists options used
only in the top level, while the `recursive' variant lists options
also present in any nested packages.
`--version'
`-V'
Print the version of Autoconf used to generate the `configure'
script, and exit.
`--cache-file=FILE'
Enable the cache: use and save the results of the tests in FILE,
traditionally `config.cache'. FILE defaults to `/dev/null' to
disable caching.
`--config-cache'
`-C'
Alias for `--cache-file=config.cache'.
`--quiet'
`--silent'
`-q'
Do not print messages saying which checks are being made. To
suppress all normal output, redirect it to `/dev/null' (any error
messages will still be shown).
`--srcdir=DIR'
Look for the package's source code in directory DIR. Usually
`configure' can determine that directory automatically.
`--prefix=DIR'
Use DIR as the installation prefix. *note Installation Names::
for more details, including other options available for fine-tuning
the installation locations.
`--no-create'
`-n'
Run the configure checks, but stop before creating any output
files.
`configure' also accepts some other, not widely useful, options. Run
`configure --help' for more details.

Some files were not shown because too many files have changed in this diff Show more