feat: add basic fuzz testing#1896
Conversation
Signed-off-by: Antonio Ceppellini <antonio.ceppellini@gmail.com>
Signed-off-by: Antonio Ceppellini <antonio.ceppellini@gmail.com>
Codecov Report✅ All modified and coverable lines are covered by tests. @@ Coverage Diff @@
## main #1896 +/- ##
==========================================
+ Coverage 93.66% 93.68% +0.02%
==========================================
Files 144 144
Lines 9348 9348
==========================================
+ Hits 8756 8758 +2
+ Misses 592 590 -2 🚀 New features to boost your workflow:
|
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughAdds Hypothesis-based fuzz testing: dev dependency and pytest marker, Changes
Sequence Diagram(s)(Skipped — changes add test infra and tests; no new multi-component sequential runtime flow requiring visualization.) Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. 📋 Issue PlannerBuilt with CodeRabbit's Coding Plans for faster development and fewer bugs. View plan used: ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 12
ℹ️ Review info
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (11)
.gitignoreCHANGELOG.mdpyproject.tomlpytest.initests/fuzz/__init__.pytests/fuzz/conftest.pytests/fuzz/test_contract_function_parameters_fuzz.pytests/fuzz/test_entity_id_fuzz.pytests/fuzz/test_hbar_fuzz.pytests/fuzz/test_keys_fuzz.pytests/fuzz/test_transaction_from_bytes_fuzz.py
|
Great work introducing fuzz testing to the Python SDK, @AntonioCeppellini! This is a valuable addition that will help catch edge cases we might miss with traditional unit tests. A few thoughts on the failing Hbar tests: On the failing tests ( The assertion failure at Options to consider:
Given this is testing money-related values, I'd lean toward option 1 (document and constrain the input range) unless there's a specific requirement to handle values beyond float64 precision. Minor suggestions:
Excellent initiative on bringing property-based testing to the SDK! This will definitely improve our confidence in edge case handling. 🎯 |
Signed-off-by: Antonio Ceppellini <antonio.ceppellini@gmail.com>
There was a problem hiding this comment.
♻️ Duplicate comments (1)
.gitignore (1)
34-35:⚠️ Potential issue | 🟡 MinorFix the Hypothesis cache directory typo.
.hypotesis/is misspelled and will not ignore Hypothesis state artifacts. Use.hypothesis/instead.Suggested patch
# Hypothesis state -.hypotesis/ +.hypothesis/
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 8becebd3-8304-4040-90be-e0631e79e5ca
📒 Files selected for processing (1)
.gitignore
Signed-off-by: Antonio Ceppellini <antonio.ceppellini@gmail.com>
|
Hi, this is WorkflowBot.
|
There was a problem hiding this comment.
I think the tests are very good ..for the errors I believe that we should ban floats from internal representation and return decimal instead of float in conversions, we should implement this change to ensure the tests pass, and possibly open a new issue for it.
|
Hello, this is the OfficeHourBot. This is a reminder that the Hiero Python SDK Office Hours are scheduled in approximately 4 hours (14:00 UTC). This session provides an opportunity to ask questions regarding this Pull Request. Details:
Disclaimer: This is an automated reminder. Please verify the schedule here for any changes. From, |
…id float precision loss Signed-off-by: Antonio Ceppellini <antonio.ceppellini@gmail.com>
Signed-off-by: Antonio Ceppellini <antonio.ceppellini@gmail.com>
Signed-off-by: Antonio Ceppellini <antonio.ceppellini@gmail.com>
There was a problem hiding this comment.
Actionable comments posted: 1
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 04368ca4-1e51-490e-ae22-d3c718f4fc55
📒 Files selected for processing (2)
CHANGELOG.mdtests/unit/hbar_test.py
|
@MonaaEid @exploreriii i have made a small change in file |
The changelog is not updated this is the old version.. the assert warning I think we should ignore the B101 it in the pyprojet.toml |
exploreriii
left a comment
There was a problem hiding this comment.
Hi @AntonioCeppellini
Well done with this
Problem this is so difficult to review, for many reasons
Could we settle on a middle ground where the conftest in particular is made a bit more user friendly, which will help us expand this in the future as we add more tests?
|
Ok so i know this is basic fuzz testing - but we will need it to be solid enough for others to build from maybe something like this can be more maintaiable: and maybe eg Later we could try:
|
Signed-off-by: Antonio Ceppellini <antonio.ceppellini@gmail.com>
Signed-off-by: Antonio Ceppellini <antonio.ceppellini@gmail.com>
|
Hi @AntonioCeppellini could you please rebase, there was a tck issue on accounts that got merged which may have touched similar areas to you |
exploreriii
left a comment
There was a problem hiding this comment.
Love it! So much easier to maintain and understand
Just final tidy please
Signed-off-by: Antonio Ceppellini <antonio.ceppellini@gmail.com>
Signed-off-by: Antonio Ceppellini <antonio.ceppellini@gmail.com>
Signed-off-by: Antonio Ceppellini <antonio.ceppellini@gmail.com>
Description:
This PR adds:
hypothesis package (docs: https://hypothesis.readthedocs.io/en/latest/quickstart.html) to the sdk,
basic fuzz testing implementation, introduce the new
tests/fuzz/folder with:__init__.pyfileconftest.pyfileRelated issue(s):
Fixes #1872
Important notes for reviewer:
First time doing fuzz testing ULTRA open to any kind of suggestion.
Two fuzz tests are failing here:
========================================================================================== FAILURES ========================================================================================== _________________________________________________________________________ test_hbar_from_string_accepts_valid_inputs _________________________________________________________________________ @given(case=get_strategy("hbar_valid_string")) > def test_hbar_from_string_accepts_valid_inputs(case: HbarStringCase) -> None: ^^^ tests/fuzz/test_hbar_fuzz.py:22: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ tests/fuzz/test_hbar_fuzz.py:25: in test_hbar_from_string_accepts_valid_inputs _assert_hbar_invariants(parsed, case.tinybars) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ value = Hbar(90071992.54740994), expected_tinybars = 9007199254740993 def _assert_hbar_invariants(value: Hbar, expected_tinybars: int) -> None: > assert value.to_tinybars() == expected_tinybars E assert 9007199254740992 == 9007199254740993 E + where 9007199254740992 = to_tinybars() E + where to_tinybars = Hbar(90071992.54740994).to_tinybars E Falsifying example: test_hbar_from_string_accepts_valid_inputs( E case=(lambda tinybars: _hbar_string_case(HbarUnit.HBAR, tinybars))( E 9_007_199_254_740_993, E ), E ) tests/fuzz/test_hbar_fuzz.py:14: AssertionError ________________________________________________________________________ test_hbar_constructor_preserves_exact_value _________________________________________________________________________ @given(case=get_strategy("hbar_valid_constructor")) > def test_hbar_constructor_preserves_exact_value(case: HbarConstructorCase) -> None: ^^^ tests/fuzz/test_hbar_fuzz.py:36: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ tests/fuzz/test_hbar_fuzz.py:39: in test_hbar_constructor_preserves_exact_value _assert_hbar_invariants(parsed, case.tinybars) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ value = Hbar(90071992.54740994), expected_tinybars = 9007199254740993 def _assert_hbar_invariants(value: Hbar, expected_tinybars: int) -> None: > assert value.to_tinybars() == expected_tinybars E assert 9007199254740992 == 9007199254740993 E + where 9007199254740992 = to_tinybars() E + where to_tinybars = Hbar(90071992.54740994).to_tinybars E Falsifying example: test_hbar_constructor_preserves_exact_value( E case=(lambda tinybars: _hbar_constructor_case(HbarUnit.TINYBAR, tinybars))( E 9_007_199_254_740_993, E ), E ) tests/fuzz/test_hbar_fuzz.py:14: AssertionError ================================================================================== short test summary info =================================================================================== FAILED tests/fuzz/test_hbar_fuzz.py::test_hbar_from_string_accepts_valid_inputs - assert 9007199254740992 == 9007199254740993 FAILED tests/fuzz/test_hbar_fuzz.py::test_hbar_constructor_preserves_exact_value - assert 9007199254740992 == 9007199254740993 =============================================================================== 2 failed, 35 passed in 27.87s ================================================================================and this could be good because it could be a bug and we found it or, maybe I could make a softer check because i'm checking quite extreme values.
BUT we are a losing right now SMAAAAAAAAAAAAAALLLLLLLLLLLLLLLLL values and it could be a HUUUUUGE problem because we are talking about money.
To resolve the failing tests i applied a small change in hbar file
FINAL CONSIDERATION
The two failing tests could be my errors :D
Checklist