False negatives of Basilisk

[sharkdp]

Here are some examples of things that major other type checkers (mypy, pyright, pyrefly, ty) catch, but Basilisk does not.

1. Assigning a string to an integer variable.

   x: int
   x = "foo"

2. Calling an integer.

   nonsense = 123
   nonsense()

3. Putting a string into a list of integers.

   numbers: list[int] = []
   numbers[0] = "three"

4. Using an integer key in a string-keyed dictionary.

   config: dict[str, int] = {}
   config[0] = 3

5. Passing a string to a method that expects an integer.

   class C:
       def square(self, x: int) -> int:
           return x * x
   
   c = C()
   c.square("hello")

6. Calling len with the wrong number of arguments.

   len()
   len([], 1)

7. Missing a required integer return value.

   def f() -> int:
       print("hello")

8. Awaiting an integer.

   async def main() -> None:
       await 1

9. Unpacking an integer.

   a, b = 1

10. Assigning a nonexistent attribute on object.

    obj = object()
    obj.non_existing = 1

11. Indexing a list with a string.

    numbers: list[int] = []
    numbers["zero"] = 3

12. Storing a string in an integer-valued dictionary.

    config: dict[str, int] = {}
    config["retries"] = "three"

13. Unpacking too many values.

    a, b = (1, 2, 3)

14. Unpacking too few values.

    a, b = (1,)

15. Iterating over an integer.

    nonsense = 123
    
    for x in nonsense:
        pass

16. Passing an integer to json.loads.

    import json
    
    json.loads(5)

17. Passing a string to an integer keyword parameter.

    def foo(x: int, y: int, *, z: int = 0) -> int:
        return x * y * z
    
    foo(1, 2, z="hello")

18. Using an object without context-manager methods in with.

    class Manager:
        ...
    
    with Manager():
        pass

19. Indexing an object without __getitem__.

    class NotSubscriptable:
        ...
    
    a = NotSubscriptable()[0]

20. Assigning through a subscript without __setitem__.

    class NoSetitem:
        ...
    
    a = NoSetitem()
    a[0] = 0

21. Assigning a string to an integer through a walrus expression.

    x: int
    (x := "three")

22. Declaring an existing integer variable as a string.

    x = 1
    x: str

23. Replacing a class with an integer.

    class C:
        ...
    
    C = 1

24. Calling type without arguments.

    type()

25. Passing a string among integer variadic arguments.

    def foo(*numbers: int) -> int:
        return len(numbers)
    
    foo(1, 2, 3, "hello", 5)

26. Passing a string among integer keyword arguments.

    def foo(**numbers: int) -> int:
        return len(numbers)
    
    foo(a=1, b=2, c=3, d="hello", e=5)

27. Assigning to a read-only property.

    class DontAssignToMe:
        @property
        def immutable(self):
            ...
    
    DontAssignToMe().immutable = "changed"

28. Using yield from on an integer.

    from typing import Generator
    
    def generator() -> Generator[None]:
        yield from 42

29. Using invalid string slice bounds.

    def invalid_slice_bounds(s: str, start: float, end: float) -> str:
        return s[start:end]
    
    "foo"["bar":"baz"]

30. Using async with on an object without async context-manager methods.

    class Manager:
        ...
    
    async def main():
        async with Manager():
            pass

[MelbourneDeveloper]

Thanks for the report @sharkdp. The whole project is now reorienting towards removing any code that provides false negatives and shelling out to external tools where accuracy can't be achieved.