diff --git a/README.md b/README.md
index 86a8666..9acaeca 100644
--- a/README.md
+++ b/README.md
@@ -1,7 +1,7 @@
# tinytoml
[](https://github.com/FourierTransformer/tinytoml/actions/workflows/test-and-coverage.yml) [](https://coveralls.io/github/FourierTransformer/tinytoml?branch=main)
-tinytoml is a pure Lua [TOML](https://toml.io) parsing library. It's written in [Teal](https://github.com/teal-language/tl) and works with Lua 5.1-5.5 and LuaJIT 2.0/2.1. tinytoml parses a TOML document into a standard Lua table using default Lua types. Since TOML supports various datetime types, those are by default represented by strings, but can be configured to use a custom type if desired.
+tinytoml is a pure Lua [TOML](https://toml.io) parsing library. It's written in [Teal](https://github.com/teal-language/tl) and works with Lua 5.1-5.5 and LuaJIT 2.0/2.1. tinytoml parses a TOML document into a standard Lua table using default Lua types. Since TOML supports various datetime types, those are by default represented by strings, but can be configured as a table or passed in to a method so it is represented by a custom or 3rd-party library.
tinytoml passes all the [toml-test](https://github.com/toml-lang/toml-test) use cases that Lua can realistically pass (even the UTF-8 ones!). The few that fail are mostly representational:
- Lua doesn't differentiate between an array or a dictionary, so tests involving _empty_ arrays fail.
@@ -9,6 +9,10 @@ tinytoml passes all the [toml-test](https://github.com/toml-lang/toml-test) use
Current Supported TOML Version: 1.1.0
+> [!TIP]
+> | [Installing](#installing) | [Parsing](#parsing-toml) | [Encoding](#encoding-toml) | [Comparison](#comparison) |
+> | ---------- | ------- | -------- | ---------- |
+
## Installing
You can grab the `tinytoml.lua` file from this repo (or the `tinytoml.tl` file if using Teal) or install it via LuaRocks
@@ -68,9 +72,43 @@ There are a few parsing options available that are passed in the the `options` p
tinytoml.parse("a=2024-10-31T12:49:00Z", {load_from_string=true, type_conversion=type_conversion})
```
-- `assign_value_function`
+- `encode_datetime_as` (default `string`)
+
+ Allows encoding datetime either as a `string` or a `table`. The `table` will take all the individual fields and place them in a table.
+ This can be used in conjunction with `type_conversion` - either the string or table representation would be passed into whatever function is
+ specified in `type_conversion`.
+
+ Example:
+
+ ```toml
+ offset_datetime = 1979-05-27T07:32:00Z
+ local_datetime = 1979-05-27T07:32:00
+ local_time = 07:32:00
+ local_date = 1979-05-27
+ ```
+
+ ```lua
+ -- with the option: { encode_datetime_as = "string" }
+ {
+ offset_datetime = "1979-05-27T07:32:00Z",
+ local_datetime = "1979-05-27T07:32:00",
+ local_time = "07:32:00",
+ local_date = "1979-05-27"
+ }
+ -- with the option: { encode_datetime_as = "table" }
+ {
+ offset_datetime = {year = 1979, month = 05, day = 27, hour = 7, min = 32, sec = 0, msec = 0, time_offset = "00:00"},
+ local_datetime = {year = 1979, month = 05, day = 27, hour = 7, min = 32, sec = 0, msec = 0},
+ local_time = {hour = 7, min = 32, sec = 0, msec = 0},
+ local_date = {year = 1979, month = 05, day = 27}
+ }
+
+ ```
+
+- `max_nesting_depth` (default `1000`) and `max_filesize` (default `100000000` - 100 MB)
+
+ The maximum nesting depth and maxmimum filesize in bytes. tinytoml will throw an error if either of these are exceeded.
- this method is called when assigning _every_ value to a table. It's mostly used to help perform the unit testing using [toml-test](https://github.com/toml-lang/toml-test), since they want to see the type and parsed value for comparison purposes. This option is the only one that has potential to change, so we advice against using it. If you need specific functionality that you're implementing through this (or find this function useful in general) - please let us know.
## Encoding TOML
@@ -116,4 +154,24 @@ local_time = 07:32:00
local_date = 1979-05-27
```
-This effectively means you'll have to pre-process dates and times to strings in your codebase, before passing them to tinytoml's encoder.
\ No newline at end of file
+This effectively means you'll have to pre-process dates and times to strings in your codebase, before passing them to tinytoml's encoder.
+
+## Comparison
+Here's a helpful comparison table that can be useful in deciding which Lua TOML parser to use. The data was collected with the most recent versions as of 1/2026.
+
+| Feature / Library | tinytoml | toml-edit | toml.lua | toml2lua | tomlua |
+|:------------------|:------------------------------|:------------------------------|:------------------------------|:-------------------------------|:------------------------------|
+| Language | Lua | Rust binding | C++ binding | Lua | C |
+| TOML Version | 1.1.0 | 1.0.0 | 1.0.0 | 1.0.0 | Not Specified |
+| UTF-8 Support | ✅ | ✅ | ✅ | ✅ | ✅ |
+| Passes toml-test | ✅ | ✅ | ✅ | ❌ | ❌ |
+| Date/Time Support | String/Table/Register Method | | Custom Userdata/Lua Table | Lua Table | Custom Userdata |
+| Encoder | Basic | Comment Preserving | Basic, many options | Basic | Very Configurable |
+| 16 KB TOML decode | Lua: 3.9ms
LuaJIT: 2.7ms | Lua: 2.8ms
LuaJIT: 1.0ms | Lua: dnf
LuaJIT: 2.4ms | Lua: 32.5ms
LuaJIT: 7.0ms | Lua: 1.6ms
LuaJIT: .29ms |
+| 8 MB TOML decode | Lua: 1.49s
LuaJIT: 415ms | Lua: 929ms
LuaJIT: 462ms | Lua: error
LuaJIT: error | Lua: 12.01s
LuaJIT: 3.13s | Lua: 318ms
LuaJIT: 119.7ms |
+
+**NOTES:**
+- tinytoml, toml2lua, and tomlua's toml-test support were verified by running through toml-test. toml-edit and toml.lua were based on the bindings, which both passed toml-test.
+- I was using hyperfine to run the tests, and toml.lua's time estimate rapidly started rising in the middle of the 16KB run and segfaulted with the higher runs.
+- Tests were run in a docker container running on an arm64 Mac, as tomlua did not compile on macOS at the time the benchmarks were taken.
+- Standard benchmark disclaimer: These are all relative to each other and your mileage will [likely] vary.
diff --git a/spec/decoder.lua b/spec/decoder.lua
index 55bb63c..e74f847 100644
--- a/spec/decoder.lua
+++ b/spec/decoder.lua
@@ -28,12 +28,44 @@ local function float_to_string(x)
end
-local assign_value_function = function(value, value_type)
- if value_type == "float" then
- return { ["value"] = float_to_string(value), ["type"] = value_type }
+local function add_toml_test_tag(table_to_clear)
+ if type(table_to_clear) ~= "table" then
+
+ if type(table_to_clear) == "number" then
+ if math.type(table_to_clear) == "integer" then
+ return {type="integer", value=tostring(table_to_clear)}
+ else
+ return {type="float", value=float_to_string(table_to_clear)}
+ end
+
+ elseif type(table_to_clear) == "string" then
+ return {type="string", value=table_to_clear}
+
+ elseif type(table_to_clear) == "boolean" then
+ return {type="bool", value=tostring(table_to_clear)}
+
+ else
+ return table_to_clear["value"]
+ end
+
else
- return { ["value"] = tostring(value), ["type"] = value_type }
+ if not (table_to_clear.type and table_to_clear.value) then
+ for k, v in pairs(table_to_clear) do
+ table_to_clear[k] = add_toml_test_tag(v)
+ end
+ end
end
+
+ return table_to_clear
end
-print(cjson.encode(tinytoml.parse(io.read("*a"), { load_from_string = true, assign_value_function = assign_value_function })))
+local type_conversion = {
+ ["datetime"] = function(raw_string) return {type="datetime", value=raw_string} end,
+ ["datetime-local"] = function(raw_string) return {type="datetime-local", value=raw_string} end,
+ ["date-local"] = function(raw_string) return {type="date-local", value=raw_string} end,
+ ["time-local"] = function(raw_string) return {type="time-local", value=raw_string} end,
+}
+
+local output = tinytoml.parse(io.read("*a"), { load_from_string = true, encode_datetime_as = "string", type_conversion = type_conversion })
+add_toml_test_tag(output)
+print(cjson.encode(output))
diff --git a/spec/decoder.tl b/spec/decoder.tl
deleted file mode 100644
index 4510dc2..0000000
--- a/spec/decoder.tl
+++ /dev/null
@@ -1,40 +0,0 @@
-#!/usr/bin/env lua
-
-local cjson = require("cjson")
-local tinytoml = require("tinytoml")
-
-local to_inf_and_beyound = {
- ["inf"] = true,
- ["-inf"] = true,
- ["nan"] = true,
- ["-nan"] = true
-}
-
--- Using a slightly modified version from https://stackoverflow.com/a/69827191
-local function float_to_string(x: number): string
- -- a table key can't be nan in Lua, and I would've rather checked for equality with
- -- math.huge/nan (as 0/0), but (0/0) != (0/0) in Lua so I think this is probably fine.
- if to_inf_and_beyound[tostring(x)] then
- return tostring(x)
- end
- for precision = 15, 17 do
- -- Use a 2-layer format to try different precisions with %g.
- local s = ('%%.%dg'):format(precision):format(x)
- -- See if s is an exact representation of x.
- if tonumber(s) == x then
- return s
- end
- end
-end
-
--- the format toml-test expects
-local assign_value_function = function(value: any, value_type?: string): any
- if value_type == "float" then
- return {["value"]=float_to_string(value as number), ["type"]=value_type}
- else
- return {["value"]=tostring(value), ["type"]=value_type}
- end
-end
-
-print(cjson.encode(tinytoml.parse(io.read("*a"), {load_from_string=true, assign_value_function=assign_value_function})))
-
diff --git a/tinytoml-0.1.0-1.rockspec b/tinytoml-1.0.0-1.rockspec
similarity index 87%
rename from tinytoml-0.1.0-1.rockspec
rename to tinytoml-1.0.0-1.rockspec
index 134338d..adfc7f2 100644
--- a/tinytoml-0.1.0-1.rockspec
+++ b/tinytoml-1.0.0-1.rockspec
@@ -1,9 +1,9 @@
package = "tinytoml"
-version = "0.1.0-1"
+version = "1.0.0-1"
source = {
url = "git://github.com/FourierTransformer/tinytoml.git",
- tag = "0.1.0"
+ tag = "1.0.0"
}
description = {
@@ -13,7 +13,6 @@ description = {
It supports all TOML 1.1.0 features including parsing strings, numbers, datetimes, arrays, inline-tables and even validating UTF-8 with good error messages if anything fails!
]],
homepage = "https://github.com/FourierTransformer/tinytoml",
- maintainer = "Fourier Transformer ",
license = "MIT"
}
diff --git a/tinytoml.lua b/tinytoml.lua
index 84e5859..1a314bd 100644
--- a/tinytoml.lua
+++ b/tinytoml.lua
@@ -13,6 +13,11 @@
+
+
+
+
+
local tinytoml = {}
@@ -146,8 +151,6 @@ tinytoml._LICENSE = "MIT"
-
-
@@ -628,31 +631,73 @@ local function validate_month_date(sm, year, month, day, anchor)
end
end
+local function assign_time_local(sm, match, hour, min, sec, msec)
+ sm.value_type = "time-local"
+ if sm.options.encode_datetime_as == "string" then
+ sm.value = sm.options.type_conversion[sm.value_type](match)
+ else
+ sm.value = sm.options.type_conversion[sm.value_type]({ hour = hour, min = min, sec = sec, msec = msec })
+ end
+end
+
+local function assign_date_local(sm, match, year, month, day)
+ sm.value_type = "date-local"
+ if sm.options.encode_datetime_as == "string" then
+ sm.value = sm.options.type_conversion[sm.value_type](match)
+ else
+ sm.value = sm.options.type_conversion[sm.value_type]({ year = year, month = month, day = day })
+ end
+end
+
+local function assign_datetime_local(sm, match, year, month, day, hour, min, sec, msec)
+ sm.value_type = "datetime-local"
+ if sm.options.encode_datetime_as == "string" then
+ sm.value = sm.options.type_conversion[sm.value_type](match)
+ else
+ sm.value = sm.options.type_conversion[sm.value_type]({ year = year, month = month, day = day, hour = hour, min = min, sec = sec, msec = msec or 0 })
+ end
+end
+
+local function assign_datetime(sm, match, year, month, day, hour, min, sec, msec, tz)
+ if tz then
+ local hour_s, min_s
+ sm._, sm._, hour_s, min_s = tz:find("^[+-](%d%d):(%d%d)$")
+ validate_hours_minutes(sm, _tointeger(hour_s), _tointeger(min_s), "offset-date-time")
+ end
+ sm.value_type = "datetime"
+ if sm.options.encode_datetime_as == "string" then
+ sm.value = sm.options.type_conversion[sm.value_type](match)
+ else
+ sm.value = sm.options.type_conversion[sm.value_type]({ year = year, month = month, day = day, hour = hour, min = min, sec = sec, msec = msec or 0, time_offset = tz or "00:00" })
+ end
+end
+
local function validate_datetime(sm, value)
+ local hour_s, min_s, sec_s, msec_s
local hour, min, sec
- sm._, sm._, sm.match, hour, min, sm.ext = value:find("^((%d%d):(%d%d))(.*)$")
+ sm._, sm._, sm.match, hour_s, min_s, sm.ext = value:find("^((%d%d):(%d%d))(.*)$")
if sm.match then
- validate_hours_minutes(sm, _tointeger(hour), _tointeger(min), "local-time")
+ hour, min = _tointeger(hour_s), _tointeger(min_s)
+ validate_hours_minutes(sm, hour, min, "local-time")
if sm.ext ~= "" then
- sm._, sm._, sec = sm.ext:find("^:(%d%d)$")
- if sec then
- validate_seconds(sm, _tointeger(sec), "local-time")
- sm.value_type = "time-local"
- sm.value = sm.type_conversion[sm.value_type](sm.match .. sm.ext)
+ sm._, sm._, sec_s = sm.ext:find("^:(%d%d)$")
+ if sec_s then
+ sec = _tointeger(sec_s)
+ validate_seconds(sm, sec, "local-time")
+ assign_time_local(sm, sm.match .. sm.ext, hour, min, sec, 0)
return true
end
- sm._, sm._, sec = sm.ext:find("^:(%d%d)%.%d+$")
- if sec then
- validate_seconds(sm, _tointeger(sec), "local-time")
- sm.value_type = "time-local"
- sm.value = sm.type_conversion[sm.value_type](sm.match .. sm.ext)
+ sm._, sm._, sec_s, msec_s = sm.ext:find("^:(%d%d)%.(%d+)$")
+ if sec_s then
+ sec = _tointeger(sec_s)
+ validate_seconds(sm, sec, "local-time")
+ assign_time_local(sm, sm.match .. sm.ext, hour, min, sec, _tointeger(msec_s))
return true
end
else
- sm.value_type = "time-local"
- sm.value = sm.type_conversion[sm.value_type](sm.match .. ":00")
+ assign_time_local(sm, sm.match .. ":00", hour, min, 0, 0)
return true
end
end
@@ -663,8 +708,7 @@ local function validate_datetime(sm, value)
if sm.match then
year, month, day = _tointeger(year_s), _tointeger(month_s), _tointeger(day_s)
validate_month_date(sm, year, month, day, "local-date")
- sm.value_type = "date-local"
- sm.value = sm.type_conversion[sm.value_type](sm.match)
+ assign_date_local(sm, sm.match, year, month, day)
@@ -682,20 +726,21 @@ local function validate_datetime(sm, value)
end
end
- sm._, sm._, sm.match, year_s, month_s, day_s, hour, min, sm.ext =
+ sm._, sm._, sm.match, year_s, month_s, day_s, hour_s, min_s, sm.ext =
value:find("^((%d%d%d%d)%-(%d%d)%-(%d%d)[Tt ](%d%d):(%d%d))(.*)$")
if sm.match then
- validate_hours_minutes(sm, _tointeger(hour), _tointeger(min), "local-time")
+ hour, min = _tointeger(hour_s), _tointeger(min_s)
+ validate_hours_minutes(sm, hour, min, "local-time")
year, month, day = _tointeger(year_s), _tointeger(month_s), _tointeger(day_s)
validate_month_date(sm, year, month, day, "local-date-time")
local temp_ext
- sm._, sm._, sec, temp_ext = sm.ext:find("^:(%d%d)(.*)$")
- if sec then
- validate_seconds(sm, _tointeger(sec), "local-time")
- sm.match = sm.match .. ":" .. sec
+ sm._, sm._, sec_s, temp_ext = sm.ext:find("^:(%d%d)(.*)$")
+ if sec_s then
+ validate_seconds(sm, _tointeger(sec_s), "local-time")
+ sm.match = sm.match .. ":" .. sec_s
sm.ext = temp_ext
else
sm.match = sm.match .. ":00"
@@ -705,33 +750,29 @@ local function validate_datetime(sm, value)
if sm.ext ~= "" then
sm.match = sm.match .. sm.ext
if sm.ext:find("^%.%d+$") then
- sm.value_type = "datetime-local"
- sm.value = sm.type_conversion[sm.value_type](sm.match)
+ sm._, sm._, msec_s = sm.ext:find("^%.(%d+)Z$")
+ assign_datetime_local(sm, sm.match, year, month, day, hour, min, sec, _tointeger(msec_s))
return true
elseif sm.ext:find("^%.%d+Z$") then
- sm.value_type = "datetime"
- sm.value = sm.type_conversion[sm.value_type](sm.match)
+ sm._, sm._, msec_s = sm.ext:find("^%.(%d+)Z$")
+ assign_datetime(sm, sm.match, year, month, day, hour, min, sec, _tointeger(msec_s))
return true
elseif sm.ext:find("^%.%d+[+-]%d%d:%d%d$") then
- sm._, sm.end_seq, hour, min = sm.ext:find("^%.%d+[+-](%d%d):(%d%d)$")
- validate_hours_minutes(sm, _tointeger(hour), _tointeger(min), "offset-date-time")
- sm.value_type = "datetime"
- sm.value = sm.type_conversion[sm.value_type](sm.match)
+ local tz_s
+ sm._, sm._, msec_s, tz_s = sm.ext:find("^%.(%d+)([+-]%d%d:%d%d)$")
+ assign_datetime(sm, sm.match, year, month, day, hour, min, sec, _tointeger(msec_s), tz_s)
return true
elseif sm.ext:find("^[Zz]$") then
- sm.value_type = "datetime"
- sm.value = sm.type_conversion[sm.value_type](sm.match)
+ assign_datetime(sm, sm.match, year, month, day, hour, min, sec)
return true
elseif sm.ext:find("^[+-]%d%d:%d%d$") then
- sm._, sm.end_seq, hour, min = sm.ext:find("^[+-](%d%d):(%d%d)$")
- validate_hours_minutes(sm, _tointeger(hour), _tointeger(min), "offset-date-time")
- sm.value_type = "datetime"
- sm.value = sm.type_conversion[sm.value_type](sm.match)
+ local tz_s
+ sm._, sm._, tz_s = sm.ext:find("^([+-]%d%d:%d%d)$")
+ assign_datetime(sm, sm.match, year, month, day, hour, min, sec, 0, tz_s)
return true
end
else
- sm.value_type = "datetime-local"
- sm.value = sm.type_conversion[sm.value_type](sm.match)
+ assign_datetime_local(sm, sm.match, year, month, day, hour, min, sec)
return true
end
end
@@ -790,11 +831,7 @@ local function create_array(sm)
end
local function add_array_comma(sm)
- if sm.value_type == "array" or sm.value_type == "inline-table" then
- table.insert(sm.arrays[sm.nested_arrays], sm.value)
- else
- table.insert(sm.arrays[sm.nested_arrays], sm.assign_value_function(sm.value, sm.value_type))
- end
+ table.insert(sm.arrays[sm.nested_arrays], sm.value)
sm.value = nil
sm.i = sm.i + 1
@@ -934,11 +971,7 @@ end
local function assign_value(sm)
local output = {}
- if sm.value_type == "array" or sm.value_type == "inline-table" then
- output = sm.value
- else
- output = sm.assign_value_function(sm.value, sm.value_type)
- end
+ output = sm.value
local out_table = sm.current_table
@@ -1120,39 +1153,78 @@ local transitions = {
},
}
-local function generic_assign(value) return value end
local function generic_type_conversion(raw_value) return raw_value end
function tinytoml.parse(filename, options)
local sm = {}
- sm.assign_value_function = generic_assign
- sm.type_conversion = {
- ["datetime"] = generic_type_conversion,
- ["datetime-local"] = generic_type_conversion,
- ["date-local"] = generic_type_conversion,
- ["time-local"] = generic_type_conversion,
+
+ local default_options = {
+ max_nesting_depth = 1000,
+ max_filesize = 100000000,
+ load_from_string = false,
+ encode_datetime_as = "string",
+ type_conversion = {
+ ["datetime"] = generic_type_conversion,
+ ["datetime-local"] = generic_type_conversion,
+ ["date-local"] = generic_type_conversion,
+ ["time-local"] = generic_type_conversion,
+ },
}
if options then
- if options.load_from_string then
- sm.input = filename
- sm.filename = "string input"
+
+ if options.max_nesting_depth ~= nil then
+ assert(type(options.max_nesting_depth) == "number", "the tinytoml option 'max_nesting_depth' takes in a 'number'. You passed in the value '" .. tostring(options.max_nesting_depth) .. "' of type '" .. type(options.max_nesting_depth) .. "'")
end
- if options.assign_value_function then
- sm.assign_value_function = options.assign_value_function
- else
+
+ if options.max_filesize ~= nil then
+ assert(type(options.max_filesize) == "number", "the tinytoml option 'max_filesize' takes in a 'number'. You passed in the value '" .. tostring(options.max_filesize) .. "' of type '" .. type(options.max_filesize) .. "'")
+ end
+
+ if options.load_from_string ~= nil then
+ assert(type(options.load_from_string) == "boolean", "the tinytoml option 'load_from_string' takes in a 'function'. You passed in the value '" .. tostring(options.load_from_string) .. "' of type '" .. type(options.load_from_string) .. "'")
end
- if options.type_conversion then
+
+ if options.encode_datetime_as ~= nil then
+ assert(type(options.encode_datetime_as) == "string", "the tinytoml option 'encode_datetime_as' takes in either the 'string' or 'table' (as type 'string'). You passed in the value '" .. tostring(options.encode_datetime_as) .. "' of type '" .. type(options.encode_datetime_as) .. "'")
+ end
+
+ if options.type_conversion ~= nil then
+ assert(type(options.type_conversion) == "table", "the tinytoml option 'type_conversion' takes in a 'table'. You passed in the value '" .. tostring(options.type_conversion) .. "' of type '" .. type(options.type_conversion) .. "'")
for key, value in pairs(options.type_conversion) do
- sm.type_conversion[(key)] = value
+ assert(type(key) == "string")
+ if not default_options.type_conversion[key] then
+ error("")
+ end
+ assert(type(value) == "function")
+ end
+ end
+
+
+ options.max_nesting_depth = options.max_nesting_depth or default_options.max_nesting_depth
+ options.max_filesize = options.max_filesize or default_options.max_filesize
+ options.load_from_string = options.load_from_string or default_options.load_from_string
+ options.encode_datetime_as = options.encode_datetime_as or default_options.encode_datetime_as
+ options.type_conversion = options.type_conversion or default_options.type_conversion
+
+
+ if options.load_from_string == true then
+ sm.input = filename
+ sm.filename = "string input"
+ end
+
+
+ for key, value in pairs(default_options.type_conversion) do
+ if options.type_conversion[key] == nil then
+ options.type_conversion[key] = value
end
end
- options.max_filesize = options.max_filesize or 100000000
- options.max_nesting_depth = options.max_nesting_depth or 1000
+
else
- options = { load_from_string = false, max_filesize = 100000000, max_nesting_depth = 1000 }
+ options = default_options
end
+
sm.options = options
if options.load_from_string == false then
@@ -1272,12 +1344,14 @@ local function escape_string(str, multiline, is_key)
local sm = { input = str, i = 1, line_number = 1, line_number_char_index = 1 }
- sm.type_conversion = {
+ sm.options = {}
+ sm.options.type_conversion = {
["datetime"] = generic_type_conversion,
["datetime-local"] = generic_type_conversion,
["date-local"] = generic_type_conversion,
["time-local"] = generic_type_conversion,
}
+ sm.options.encode_datetime_as = "string"
sm._, sm.end_seq, sm.match = sm.input:find("^([^ #\r\n,%[{%]}]+)", sm.i)
diff --git a/tinytoml.tl b/tinytoml.tl
index 1a4af8c..c8fb2d6 100644
--- a/tinytoml.tl
+++ b/tinytoml.tl
@@ -5,10 +5,15 @@ local enum TomlConversionType
"time-local"
end
+local enum DateTimeEncode
+ "string"
+ "table"
+end
+
local record TinyTomlOptions
load_from_string: boolean
- assign_value_function: function(value: any, value_type?: string): any
- type_conversion: {TomlConversionType:function(raw_value: string):T}
+ type_conversion: {TomlConversionType:function(raw_value: string | table):T}
+ encode_datetime_as: DateTimeEncode
max_filesize: integer
max_nesting_depth: integer
end
@@ -147,8 +152,6 @@ local record StateMachine
meta_table: {any:{MTType}}
current_meta_table: {any:{MTType}}
inline_table_backup: {InlineTableSave} -- stores all the stuff as we descend into nested inline-tables
- assign_value_function: function(value: any, value_type?: string): any
- type_conversion: {TomlType:function(raw_value: string): T}
end
local sbyte = string.byte
@@ -628,31 +631,73 @@ local function validate_month_date(sm: StateMachine, year: integer, month: integ
end
end
+local function assign_time_local(sm: StateMachine, match: string, hour: integer, min: integer, sec: integer, msec: integer)
+ sm.value_type = "time-local"
+ if sm.options.encode_datetime_as == "string" then
+ sm.value = sm.options.type_conversion[sm.value_type as TomlConversionType](match)
+ else
+ sm.value = sm.options.type_conversion[sm.value_type as TomlConversionType]({hour=hour, min=min, sec=sec, msec=msec})
+ end
+end
+
+local function assign_date_local(sm: StateMachine, match: string, year: integer, month: integer, day: integer)
+ sm.value_type = "date-local"
+ if sm.options.encode_datetime_as == "string" then
+ sm.value = sm.options.type_conversion[sm.value_type as TomlConversionType](match)
+ else
+ sm.value = sm.options.type_conversion[sm.value_type as TomlConversionType]({year=year, month=month, day=day})
+ end
+end
+
+local function assign_datetime_local(sm: StateMachine, match: string, year: integer, month: integer, day: integer, hour: integer, min: integer, sec: integer, msec?: integer)
+ sm.value_type = "datetime-local"
+ if sm.options.encode_datetime_as == "string" then
+ sm.value = sm.options.type_conversion[sm.value_type as TomlConversionType](match)
+ else
+ sm.value = sm.options.type_conversion[sm.value_type as TomlConversionType]({year=year, month=month, day=day, hour=hour, min=min, sec=sec, msec=msec or 0})
+ end
+end
+
+local function assign_datetime(sm: StateMachine, match: string, year: integer, month: integer, day: integer, hour: integer, min: integer, sec: integer, msec?: integer, tz?: string)
+ if tz then
+ local hour_s, min_s: string, string
+ sm._, sm._, hour_s, min_s = tz:find("^[+-](%d%d):(%d%d)$")
+ validate_hours_minutes(sm, _tointeger(hour_s), _tointeger(min_s), "offset-date-time")
+ end
+ sm.value_type = "datetime"
+ if sm.options.encode_datetime_as == "string" then
+ sm.value = sm.options.type_conversion[sm.value_type as TomlConversionType](match)
+ else
+ sm.value = sm.options.type_conversion[sm.value_type as TomlConversionType]({year=year, month=month, day=day, hour=hour, min=min, sec=sec, msec=msec or 0, time_offset=tz or "00:00"})
+ end
+end
+
local function validate_datetime(sm: StateMachine, value: string): boolean
- local hour, min, sec: string, string, string
- sm._, sm._, sm.match, hour, min, sm.ext = value:find("^((%d%d):(%d%d))(.*)$") as (integer, integer, string, string, string, string)
+ local hour_s, min_s, sec_s, msec_s: string, string, string, string
+ local hour, min, sec: integer, integer, integer
+ sm._, sm._, sm.match, hour_s, min_s, sm.ext = value:find("^((%d%d):(%d%d))(.*)$") as (integer, integer, string, string, string, string)
if sm.match then
- validate_hours_minutes(sm, _tointeger(hour), _tointeger(min), "local-time")
+ hour, min = _tointeger(hour_s), _tointeger(min_s)
+ validate_hours_minutes(sm, hour, min, "local-time")
if sm.ext ~= "" then
- sm._, sm._, sec = sm.ext:find("^:(%d%d)$")
- if sec then
- validate_seconds(sm, _tointeger(sec), "local-time")
- sm.value_type = "time-local"
- sm.value = sm.type_conversion[sm.value_type](sm.match .. sm.ext)
+ sm._, sm._, sec_s = sm.ext:find("^:(%d%d)$")
+ if sec_s then
+ sec = _tointeger(sec_s)
+ validate_seconds(sm, sec, "local-time")
+ assign_time_local(sm, sm.match .. sm.ext, hour, min, sec, 0)
return true
end
- sm._, sm._, sec = sm.ext:find("^:(%d%d)%.%d+$")
- if sec then
- validate_seconds(sm, _tointeger(sec), "local-time")
- sm.value_type = "time-local"
- sm.value = sm.type_conversion[sm.value_type](sm.match .. sm.ext)
+ sm._, sm._, sec_s, msec_s = sm.ext:find("^:(%d%d)%.(%d+)$")
+ if sec_s then
+ sec = _tointeger(sec_s)
+ validate_seconds(sm, sec, "local-time")
+ assign_time_local(sm, sm.match .. sm.ext, hour, min, sec, _tointeger(msec_s))
return true
end
else
- sm.value_type = "time-local"
- sm.value = sm.type_conversion[sm.value_type](sm.match .. ":00") -- assume :00 if no seconds are provided
+ assign_time_local(sm, sm.match .. ":00", hour, min, 0, 0) -- assume :00 if no seconds are provided
return true
end
end
@@ -663,8 +708,7 @@ local function validate_datetime(sm: StateMachine, value: string): boolean
if sm.match then
year, month, day = _tointeger(year_s), _tointeger(month_s), _tointeger(day_s)
validate_month_date(sm, year, month, day, "local-date")
- sm.value_type = "date-local"
- sm.value = sm.type_conversion[sm.value_type](sm.match)
+ assign_date_local(sm, sm.match, year, month, day)
-- small hack that allows datetimes with spaces to work
-- does involve updating the current position in a validator...
@@ -682,20 +726,21 @@ local function validate_datetime(sm: StateMachine, value: string): boolean
end
end
- sm._, sm._, sm.match, year_s, month_s, day_s, hour, min, sm.ext =
+ sm._, sm._, sm.match, year_s, month_s, day_s, hour_s, min_s, sm.ext =
value:find("^((%d%d%d%d)%-(%d%d)%-(%d%d)[Tt ](%d%d):(%d%d))(.*)$")
as (integer, integer, string, string, string, string, string, string, string)
if sm.match then
- validate_hours_minutes(sm, _tointeger(hour), _tointeger(min), "local-time")
+ hour, min = _tointeger(hour_s), _tointeger(min_s)
+ validate_hours_minutes(sm, hour, min, "local-time")
year, month, day = _tointeger(year_s), _tointeger(month_s), _tointeger(day_s)
validate_month_date(sm, year, month, day, "local-date-time")
-- see if seconds are next
local temp_ext: string
- sm._, sm._, sec, temp_ext = sm.ext:find("^:(%d%d)(.*)$") as (integer, integer, string, string)
- if sec then
- validate_seconds(sm, _tointeger(sec), "local-time")
- sm.match = sm.match .. ":" .. sec
+ sm._, sm._, sec_s, temp_ext = sm.ext:find("^:(%d%d)(.*)$") as (integer, integer, string, string)
+ if sec_s then
+ validate_seconds(sm, _tointeger(sec_s), "local-time")
+ sm.match = sm.match .. ":" .. sec_s
sm.ext = temp_ext
else
sm.match = sm.match .. ":00"
@@ -705,33 +750,29 @@ local function validate_datetime(sm: StateMachine, value: string): boolean
if sm.ext ~= "" then
sm.match = sm.match .. sm.ext
if sm.ext:find("^%.%d+$") then
- sm.value_type = "datetime-local"
- sm.value = sm.type_conversion[sm.value_type](sm.match)
+ sm._, sm._, msec_s = sm.ext:find("^%.(%d+)Z$")
+ assign_datetime_local(sm, sm.match, year, month, day, hour, min, sec, _tointeger(msec_s))
return true
elseif sm.ext:find("^%.%d+Z$") then
- sm.value_type = "datetime"
- sm.value = sm.type_conversion[sm.value_type](sm.match)
+ sm._, sm._, msec_s = sm.ext:find("^%.(%d+)Z$")
+ assign_datetime(sm, sm.match, year, month, day, hour, min, sec, _tointeger(msec_s))
return true
elseif sm.ext:find("^%.%d+[+-]%d%d:%d%d$") then
- sm._, sm.end_seq, hour, min = sm.ext:find("^%.%d+[+-](%d%d):(%d%d)$") as (integer, integer, string, string)
- validate_hours_minutes(sm, _tointeger(hour), _tointeger(min), "offset-date-time")
- sm.value_type = "datetime"
- sm.value = sm.type_conversion[sm.value_type](sm.match)
+ local tz_s: string
+ sm._, sm._, msec_s, tz_s = sm.ext:find("^%.(%d+)([+-]%d%d:%d%d)$") as (integer, integer, string, string)
+ assign_datetime(sm, sm.match, year, month, day, hour, min, sec, _tointeger(msec_s), tz_s)
return true
elseif sm.ext:find("^[Zz]$") then
- sm.value_type = "datetime"
- sm.value = sm.type_conversion[sm.value_type](sm.match)
+ assign_datetime(sm, sm.match, year, month, day, hour, min, sec)
return true
elseif sm.ext:find("^[+-]%d%d:%d%d$") then
- sm._, sm.end_seq, hour, min = sm.ext:find("^[+-](%d%d):(%d%d)$") as (integer, integer, string, string)
- validate_hours_minutes(sm, _tointeger(hour), _tointeger(min), "offset-date-time")
- sm.value_type = "datetime"
- sm.value = sm.type_conversion[sm.value_type](sm.match)
+ local tz_s: string
+ sm._, sm._, tz_s = sm.ext:find("^([+-]%d%d:%d%d)$") as (integer, integer, string, string)
+ assign_datetime(sm, sm.match, year, month, day, hour, min, sec, 0, tz_s)
return true
end
else
- sm.value_type = "datetime-local"
- sm.value = sm.type_conversion[sm.value_type](sm.match)
+ assign_datetime_local(sm, sm.match, year, month, day, hour, min, sec)
return true
end
end
@@ -790,11 +831,7 @@ local function create_array(sm: StateMachine)
end
local function add_array_comma(sm: StateMachine)
- if sm.value_type == "array" or sm.value_type=="inline-table" then
- table.insert(sm.arrays[sm.nested_arrays], sm.value)
- else
- table.insert(sm.arrays[sm.nested_arrays], sm.assign_value_function(sm.value, sm.value_type))
- end
+ table.insert(sm.arrays[sm.nested_arrays], sm.value)
sm.value = nil -- to handle the post comma condition
sm.i = sm.i + 1
@@ -934,11 +971,7 @@ end
local function assign_value(sm: StateMachine)
local output = {}
- if sm.value_type == "array" or sm.value_type == "inline-table" then
- output = sm.value as {any}
- else
- output = sm.assign_value_function(sm.value, sm.value_type) as {any}
- end
+ output = sm.value as {any}
-- iterate over keys, updating the pointer at "out_table"
local out_table = sm.current_table as {any:any}
@@ -1120,39 +1153,78 @@ local transitions: {states:{integer:{function, states}}} = {
}
}
-local function generic_assign(value: any): any return value end
local function generic_type_conversion(raw_value: string): T return raw_value as T end
function tinytoml.parse(filename: string, options?: TinyTomlOptions): {string:any}
local sm: StateMachine = {}
- sm.assign_value_function = generic_assign
- sm.type_conversion = {
- ["datetime"] = generic_type_conversion,
- ["datetime-local"] = generic_type_conversion,
- ["date-local"] = generic_type_conversion,
- ["time-local"] = generic_type_conversion
+
+ local default_options: TinyTomlOptions = {
+ max_nesting_depth = 1000,
+ max_filesize = 100000000,
+ load_from_string = false,
+ encode_datetime_as = "string",
+ type_conversion = {
+ ["datetime"] = generic_type_conversion,
+ ["datetime-local"] = generic_type_conversion,
+ ["date-local"] = generic_type_conversion,
+ ["time-local"] = generic_type_conversion
+ }
}
-
+
if options then
- if options.load_from_string then
- sm.input = filename
- sm.filename = "string input"
+ -- type check all the options!
+ if options.max_nesting_depth ~= nil then
+ assert(type(options.max_nesting_depth) == "number", "the tinytoml option 'max_nesting_depth' takes in a 'number'. You passed in the value '" .. tostring(options.max_nesting_depth) .. "' of type '" .. type(options.max_nesting_depth) .. "'")
end
- if options.assign_value_function then
- sm.assign_value_function = options.assign_value_function
- else
+
+ if options.max_filesize ~= nil then
+ assert(type(options.max_filesize) == "number", "the tinytoml option 'max_filesize' takes in a 'number'. You passed in the value '" .. tostring(options.max_filesize) .. "' of type '" .. type(options.max_filesize) .. "'")
+ end
+
+ if options.load_from_string ~= nil then
+ assert(type(options.load_from_string) == "boolean", "the tinytoml option 'load_from_string' takes in a 'function'. You passed in the value '" .. tostring(options.load_from_string) .. "' of type '" .. type(options.load_from_string) .. "'")
end
- if options.type_conversion then
+
+ if options.encode_datetime_as ~= nil then
+ assert(type(options.encode_datetime_as) == "string", "the tinytoml option 'encode_datetime_as' takes in either the 'string' or 'table' (as type 'string'). You passed in the value '" .. tostring(options.encode_datetime_as) .. "' of type '" .. type(options.encode_datetime_as) .. "'")
+ end
+
+ if options.type_conversion ~= nil then
+ assert(type(options.type_conversion) == "table", "the tinytoml option 'type_conversion' takes in a 'table'. You passed in the value '" .. tostring(options.type_conversion) .. "' of type '" .. type(options.type_conversion) .. "'")
for key, value in pairs(options.type_conversion) do
- sm.type_conversion[(key as TomlType)] = value
+ assert(type(key) == "string")
+ if not default_options.type_conversion[key] then
+ error("")
+ end
+ assert(type(value) == "function")
end
end
- options.max_filesize = options.max_filesize or 100000000
- options.max_nesting_depth = options.max_nesting_depth or 1000
+
+ -- put in the defaults
+ options.max_nesting_depth = options.max_nesting_depth or default_options.max_nesting_depth
+ options.max_filesize = options.max_filesize or default_options.max_filesize
+ options.load_from_string = options.load_from_string or default_options.load_from_string
+ options.encode_datetime_as = options.encode_datetime_as or default_options.encode_datetime_as
+ options.type_conversion = options.type_conversion or default_options.type_conversion
+
+ -- verify/setup the last couple of things
+ if options.load_from_string == true then
+ sm.input = filename
+ sm.filename = "string input"
+ end
+
+ -- ensure a value is set for all the options.type_conversion
+ for key, value in pairs(default_options.type_conversion) do
+ if options.type_conversion[key] == nil then
+ options.type_conversion[key] = value
+ end
+ end
+
else
- options = {load_from_string = false, max_filesize = 100000000, max_nesting_depth = 1000}
+ options = default_options
end
+ -- finally set sm.options
sm.options = options
if options.load_from_string == false then
@@ -1272,12 +1344,14 @@ local function escape_string(str: string, multiline: boolean, is_key: boolean):
-- setting up a _very_ basic StateMachine
local sm: StateMachine = {input=str, i=1, line_number=1, line_number_char_index=1}
- sm.type_conversion = {
+ sm.options = {}
+ sm.options.type_conversion = {
["datetime"] = generic_type_conversion,
["datetime-local"] = generic_type_conversion,
["date-local"] = generic_type_conversion,
["time-local"] = generic_type_conversion
}
+ sm.options.encode_datetime_as = "string"
-- performing the same setup as in close_other_value, should set values correctly to run validate_datetime
sm._, sm.end_seq, sm.match = sm.input:find("^([^ #\r\n,%[{%]}]+)", sm.i)