The document property lookup is a bottleneck

[rweickelt]

I am working on a 250 pages document containing many and large tables. The performance of rinohtype is not great (2 minutes compared to < 10s with TeX) and I started to dig into possible bottlenecks. I took my document as a benchmark and ran:

python3 -m cProfile -o test-1.prof -m sphinx.cmd.build -b rinoh src build
pyprof2calltree -o test-1.prof -i test-1.callgrind -i test-1.prof

then analysed the .callgrind file with K/QCacheGrind:

We can see 18 Mio calls to the document_part property in layout.py.

500.000 calls seem to originate in get_style() in style.py:

I supposed the rest comes from recursive calls in the document/document_part property getters. Looking at layout.py we find:

    @cached
    def get_style(self, attribute, container):
        return self.get_config_value(attribute, container.document)

Although the get_style function is marked for explicit caching, the cache has only little effect. The container parameter is different on each call leading to a cache miss and therefore the container.document property lookup machinery is triggered more than necessary. Document access may be better solved with a singleton or - if there are multiple document objects - with an id + getter. That getter could then be cached easily.

Solving that bottleneck may cause a significant performance boost of about 12%. I feel not competent enough to solve this dilemma right now. I don't even know if my measurements are correct.

[brechtm]

Thanks for the analysis!

rinohtype's performance is indeed not great and has been a problem since the first release. In the past, I have spent considerable time trying to speed it up, but I didn't find many opportunities for major optimizations. As you identified, a big part of the problem is that there are a huge number of style lookups and these are difficult to speed up beyond caching (if you want to keep the current flexibility).

If you are interested into tackling this issue, I can only encourage you. However, this is not high on my priority list since there are important bugs that more urgently require attention.

Last time I looked into this (years ago), PyPy offered a non-negligible speedup over CPython, so you might want to try that for a low-effort performance boost.

Also, when you are developing a stylesheet, I recommend you do this with a minimal document that contains a representative sample of the document elements you will be using in you actual document(s).

Related: #18