Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -248,6 +248,9 @@ endif

sudo rm -f /usr/include/asm

docs/fluster.1: docs/fluster.1.md
pandoc $< -s -t man -o $@

clean: ## remove contrib temporary folder
rm -rf $(CONTRIB_DIR)

Expand Down
176 changes: 176 additions & 0 deletions docs/fluster.1
Original file line number Diff line number Diff line change
@@ -0,0 +1,176 @@
.\" Automatically generated by Pandoc 3.1.11.1
.\"
.TH "FLUSTER" "1" "" "" ""
.SH NAME
fluster \[en] Testing framework for video decoder conformance
.SH SYNOPSIS
\f[B]fluster\f[R] [\f[B]\-h\f[R]] [\f[B]\-r\f[R] \f[I]RESOURCES\f[R]]
[\f[B]\-o\f[R] \f[I]OUTPUT\f[R]] [\f[B]\-ne\f[R]] [\f[B]\-tsd\f[R]
\f[I]TEST_SUITES_DIR\f[R]]
\f[I]{list,l,run,r,download,d,reference,f}\f[R]
.SH DESCRIPTION
\f[B]fluster\f[R] is a testing framework written in Python for video
decoder conformance.
It is composed of a command\-line application that runs a number of test
suites with the supported decoders.
Its purpose is to check different decoder implementations against known
test suites with known and proven results.
It was originally designed to check the conformance of H.265/HEVC
decoders, but it also supports H.264/AVC, H.266/VVC, VP8, VP9, AV1 and
AAC.
.SH OPTIONS
.TP
\f[B]\-h\f[R], \f[B]\-\-help\f[R]
Show help message and exit.
.TP
\f[B]\-r\f[R] \f[I]RESOURCES\f[R], \f[B]\-\-resources\f[R] \f[I]RESOURCES\f[R]
Set the directory where resources are taken from.
.TP
\f[B]\-o\f[R] \f[I]OUTPUT\f[R], \f[B]\-\-output\f[R] \f[I]OUTPUT\f[R]
Set the directory where test results will be stored.
.TP
\f[B]\-ne\f[R] \f[I]OUTPUT\f[R], \f[B]\-\-no\-emoji\f[R]
Set to use plain text instead of emojis.
.TP
\f[B]\-tsd\f[R] \f[I]TEST_SUITES_DIR\f[R], \f[B]\-\-test\-suites\-dir\f[R] \f[I]TEST_SUITES_DIR\f[R]
Set the directory where test suite will be read from.
.SH FLUSTER COMMANDS
.TP
\f[B]list\f[R] \f[B](l)\f[R]
Show a list of available test suites and decoders.
.TP
\f[B]run\f[R] \f[B](r)\f[R]
Run test suites for decoders.
.RS
.TP
Options:
.TP
\f[B]\-j\f[R] \f[I]JOBS\f[R], \f[B]\-\-jobs\f[R] \f[I]JOBS\f[R]
Number of parallel jobs to use.
1x logical cores by default.
0 means all logical cores.
.TP
\f[B]\-t\f[R] \f[I]TIMEOUT\f[R], \f[B]\-\-timeout\f[R] \f[I]TIMEOUT\f[R]
Timeout in secs for each decoding.
Defaults to 30 secs.
.TP
\f[B]\-ff\f[R], \f[B]\-\-failfast\f[R]
Stop after first fail.
.TP
\f[B]\-q\f[R], \f[B]\-\-quiet\f[R]
Don\[cq]t show every test run.
.TP
\f[B]\-ts\f[R] \f[I]TESTSUITES\f[R], \f[B]\-\-testsuites\f[R] \f[I]TESTSUITES\f[R]
Run only the specific test suites.
.TP
\f[B]\-tv\f[R] \f[I]TESTVECTORS\f[R], \f[B]\-\-testvectors\f[R] \f[I]TESTVECTORS\f[R]
Run only the specific test vectors.
.TP
\f[B]\-d\f[R] \f[I]DECODERS\f[R], \f[B]\-\-decoders\f[R] \f[I]DECODERS\f[R]
Run only the specific decoders.
.TP
\f[B]\-s\f[R], \f[B]\-\-summary\f[R]
Generate a summary in Markdown format for each test suite.
.TP
\f[B]\-so\f[R] \f[I]SUMMARY_OUTPUT\f[R], \f[B]\-\-summary\-output\f[R] \f[I]SUMMARY_OUTPUT\f[R]
Dump summary output to file.
.TP
\f[B]\-f\f[R] \f[I]{md,csv,junitxml}\f[R], \f[B]\-\-format\f[R] \f[I]{md,csv,junitxml}\f[R]
Specify the format for the summary file.
.TP
\f[B]\-k\f[R], \f[B]\-\-keep\f[R]
Keep output files generated during the test.
.TP
\f[B]\-th\f[R] \f[I]THRESHOLD\f[R], \f[B]\-\-threshold\f[R] \f[I]THRESHOLD\f[R]
Set exit code to 2 if threshold tests are not success.
Exit code is 0 otherwise.
.TP
\f[B]\-tth\f[R] \f[I]TIME_THRESHOLD\f[R], \f[B]\-\-time\-threshold\f[R] \f[I]TIME_THRESHOLD\f[R]
Set exit code to 3 if test suite takes longer than threshold seconds.
Exit code is 0 otherwise.
.TP
\f[B]\-v\f[R], \f[B]\-\-verbose\f[R]
Show stdout and stderr of commands executed.
.RE
.TP
\f[B]download\f[R] \f[B](d)\f[R] \f[I]testsuites\f[R]
Downloads test suites resources.
.RS
.TP
Arguments:
\f[I]testsuites\f[R] List of testsuites to download.
Defaults to all.
.TP
Options:
.TP
\f[B]\-j\f[R] \f[I]JOBS\f[R], \f[B]\-\-jobs\f[R] \f[I]JOBS\f[R]
Number of parallel jobs to use.
2x logical cores by default.
0 means all logical cores.
.TP
\f[B]\-k\f[R], \f[B]\-\-keep\f[R]
Keep original downloaded file after extracting.
Only applicable to compressed files such as .zip, .tar.gz, etc
.RE
.TP
\f[B]reference\f[R] \f[B](r)\f[R]
Use a specific decoder to set its results for the test suites given.
.RS
.TP
Options:
.TP
\f[B]\-j\f[R] \f[I]JOBS\f[R], \f[B]\-\-jobs\f[R] \f[I]JOBS\f[R]
Number of parallel jobs to use.
1x logical cores by default.
0 means all logical cores.
.TP
\f[B]\-t\f[R] \f[I]TIMEOUT\f[R], \f[B]\-\-timeout\f[R] \f[I]TIMEOUT\f[R]
Timeout in secs for each decoding.
Defaults to 30 secs.
.TP
\f[B]\-ff\f[R], \f[B]\-\-failfast\f[R]
Stop after first fail.
.TP
\f[B]\-q\f[R], \f[B]\-\-quiet\f[R]
Don\[cq]t show every test run.
.TP
\f[B]\-ts\f[R] \f[I]TESTSUITES\f[R], \f[B]\-\-testsuites\f[R] \f[I]TESTSUITES\f[R]
Run only the specific test suites.
.TP
\f[B]\-tv\f[R] \f[I]TESTVECTORS\f[R], \f[B]\-\-testvectors\f[R] \f[I]TESTVECTORS\f[R]
Run only the specific test vectors.
.TP
\f[B]\-d\f[R] \f[I]DECODERS\f[R], \f[B]\-\-decoders\f[R] \f[I]DECODERS\f[R]
Run only the specific decoders.
.TP
\f[B]\-s\f[R], \f[B]\-\-summary\f[R]
Generate a summary in Markdown format for each test suite.
.TP
\f[B]\-so\f[R] \f[I]SUMMARY_OUTPUT\f[R], \f[B]\-\-summary\-output\f[R] \f[I]SUMMARY_OUTPUT\f[R]
Dump summary output to file.
.TP
\f[B]\-f\f[R] \f[I]{md,csv,junitxml}\f[R], \f[B]\-\-format\f[R] \f[I]{md,csv,junitxml}\f[R]
Specify the format for the summary file.
.TP
\f[B]\-k\f[R], \f[B]\-\-keep\f[R]
Keep output files generated during the test.
.TP
\f[B]\-th\f[R] \f[I]THRESHOLD\f[R], \f[B]\-\-threshold\f[R] \f[I]THRESHOLD\f[R]
Set exit code to 2 if threshold tests are not success.
Exit code is 0 otherwise.
.TP
\f[B]\-tth\f[R] \f[I]TIME_THRESHOLD\f[R], \f[B]\-\-time\-threshold\f[R] \f[I]TIME_THRESHOLD\f[R]
Set exit code to 3 if test suite takes longer than threshold seconds.
Exit code is 0 otherwise.
.TP
\f[B]\-v\f[R], \f[B]\-\-verbose\f[R]
Show stdout and stderr of commands executed.
.RE
.SH AUTHORS
fluster is developed by Pablo Marcos Oltra, Andoni Morales Alastruey and
contributors.
.PP
This manual page was written by Christopher Obbard \c
.MT obbardc@debian.org
.ME \c
\ for the Debian GNU/Linux system (but may be used by others).
160 changes: 160 additions & 0 deletions docs/fluster.1.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,160 @@
% FLUSTER(1)

<!---
To view the generated manpage:

$ pandoc debian/fluster.1.md -s -t man | man -l -
-->

# NAME

fluster – Testing framework for video decoder conformance

# SYNOPSIS

**fluster** [**-h**] [**-r** *RESOURCES*] [**-o** *OUTPUT*] [**-ne**] [**-tsd** *TEST_SUITES_DIR*] *{list,l,run,r,download,d,reference,f}*

# DESCRIPTION

**fluster** is a testing framework written in Python for video decoder
conformance. It is composed of a command-line application that runs a number of
test suites with the supported decoders. Its purpose is to check different
decoder implementations against known test suites with known and proven results.
It was originally designed to check the conformance of H.265/HEVC decoders, but
it also supports H.264/AVC, H.266/VVC, VP8, VP9, AV1 and AAC.

# OPTIONS

**\-h**, **\-\-help**
: Show help message and exit.

**\-r** *RESOURCES*, **\-\-resources** *RESOURCES*
: Set the directory where resources are taken from.

**\-o** *OUTPUT*, **\-\-output** *OUTPUT*
: Set the directory where test results will be stored.

**\-ne** *OUTPUT*, **\-\-no\-emoji**
: Set to use plain text instead of emojis.

**\-tsd** *TEST_SUITES_DIR*, **\-\-test\-suites\-dir** *TEST_SUITES_DIR*
: Set the directory where test suite will be read from.

# FLUSTER COMMANDS

**list** **\(l\)**
: Show a list of available test suites and decoders.

**run** **\(r\)**
: Run test suites for decoders.

Options:
: **\-j** *JOBS*, **\-\-jobs** *JOBS*
: Number of parallel jobs to use. 1x logical cores by default. 0 means all logical cores.

: **\-t** *TIMEOUT*, **\-\-timeout** *TIMEOUT*
: Timeout in secs for each decoding. Defaults to 30 secs.

: **\-ff**, **\-\-failfast**
: Stop after first fail.

: **\-q**, **\-\-quiet**
: Don't show every test run.

: **\-ts** *TESTSUITES*, **\-\-testsuites** *TESTSUITES*
: Run only the specific test suites.

: **\-tv** *TESTVECTORS*, **\-\-testvectors** *TESTVECTORS*
: Run only the specific test vectors.

: **\-d** *DECODERS*, **\-\-decoders** *DECODERS*
: Run only the specific decoders.

: **\-s**, **\-\-summary**
: Generate a summary in Markdown format for each test suite.

: **\-so** *SUMMARY_OUTPUT*, **\-\-summary\-output** *SUMMARY_OUTPUT*
: Dump summary output to file.

: **\-f** *\{md,csv,junitxml\}*, **\-\-format** *\{md,csv,junitxml\}*
: Specify the format for the summary file.

: **\-k**, **\-\-keep**
: Keep output files generated during the test.

: **\-th** *THRESHOLD*, **\-\-threshold** *THRESHOLD*
: Set exit code to 2 if threshold tests are not success. Exit code is 0 otherwise.

: **\-tth** *TIME_THRESHOLD*, **\-\-time\-threshold** *TIME_THRESHOLD*
: Set exit code to 3 if test suite takes longer than threshold seconds.
: Exit code is 0 otherwise.

: **\-v**, **\-\-verbose**
: Show stdout and stderr of commands executed.

**download** **\(d\)** *testsuites*
: Downloads test suites resources.

Arguments:
: *testsuites* List of testsuites to download. Defaults to all.

Options:
: **\-j** *JOBS*, **\-\-jobs** *JOBS*
: Number of parallel jobs to use. 2x logical cores by default. 0 means all logical cores.
: **\-k**, **\-\-keep**
: Keep original downloaded file after extracting. Only applicable to compressed files such as .zip, .tar.gz, etc

**reference** **\(r\)**
: Use a specific decoder to set its results for the test suites given.

Options:
: **\-j** *JOBS*, **\-\-jobs** *JOBS*
: Number of parallel jobs to use. 1x logical cores by default. 0 means all logical cores.

: **\-t** *TIMEOUT*, **\-\-timeout** *TIMEOUT*
: Timeout in secs for each decoding. Defaults to 30 secs.

: **\-ff**, **\-\-failfast**
: Stop after first fail.

: **\-q**, **\-\-quiet**
: Don't show every test run.

: **\-ts** *TESTSUITES*, **\-\-testsuites** *TESTSUITES*
: Run only the specific test suites.

: **\-tv** *TESTVECTORS*, **\-\-testvectors** *TESTVECTORS*
: Run only the specific test vectors.

: **\-d** *DECODERS*, **\-\-decoders** *DECODERS*
: Run only the specific decoders.

: **\-s**, **\-\-summary**
: Generate a summary in Markdown format for each test suite.

: **\-so** *SUMMARY_OUTPUT*, **\-\-summary\-output** *SUMMARY_OUTPUT*
: Dump summary output to file.

: **\-f** *\{md,csv,junitxml\}*, **\-\-format** *\{md,csv,junitxml\}*
: Specify the format for the summary file.

: **\-k**, **\-\-keep**
: Keep output files generated during the test.

: **\-th** *THRESHOLD*, **\-\-threshold** *THRESHOLD*
: Set exit code to 2 if threshold tests are not success. Exit code is 0 otherwise.

: **\-tth** *TIME_THRESHOLD*, **\-\-time\-threshold** *TIME_THRESHOLD*
: Set exit code to 3 if test suite takes longer than threshold seconds.
: Exit code is 0 otherwise.

: **\-v**, **\-\-verbose**
: Show stdout and stderr of commands executed.

# AUTHORS

fluster is developed by Pablo Marcos Oltra, Andoni Morales Alastruey and
contributors.

This manual page was written by Christopher Obbard <obbardc@debian.org>
for the Debian GNU/Linux system (but may be used by others).
3 changes: 3 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,9 @@ packages = ["fluster", "fluster.decoders"]
"test_suites/mpeg4v/MPEG4_VIDEO-SimpleScalableProfile.json",
"test_suites/mpeg4v/MPEG4_VIDEO-SimpleStudioProfile.json"
]
"share/man/man1" = [
"docs/fluster.1"
]

# Once https://github.com/pypa/pip/issues/12963 is implemented we can think about adding
# [dependency-groups]
Expand Down