Sesame 2.8.4+ memory store performance on bulk adds

Description

This issue results from a discussion on the user groups in September with James (https://groups.google.com/forum/#!topic/sesame-users/5veYxDrePUg). He asked me to capture an issue for it even though he said mentioned that:

Based on the information you have provided I am not sure if the same performance can be achieved with rollback support enabled as-is.

Apologies for only capturing it now.

To the issue: We were testing the Sesame upgrade from 2.7.15. to 2.8.4+ and noticed very slow performance in 2.8.4 and 2.8.6 when adding triples to the memory store. It's normally 2 to 3 times slower than 2.7.

After playing around with the isolation levels, it looks like the following:

  • The isolation level of NONE provides the closest performance to 2.7 but this doesn't provide any isolation so won't do for us. Also, rollback behaviour seems to be unexpected, eg. doing if you add triples, rollback, new connection, add triples then commit, you still get the triples from the "rolled back" transaction in the new connection. A rollback during concurrent updates from multiple threads also leaves the rolled back triples intact.

  • All other isolation levels (except SERIALIZABLE) are 2 times slower than 2.7.

  • SERIALIZABLE is 3-4 times slower than 2.7.

We do lots of bulk adds during a transaction that could go up to hundreds of thousands of triples per transaction. We haven't tested deletes yet.

I've attached a sample test to show the behaviour. I ran the same test against 2.7 and 2.8.4 and 2.8.6, varying the isolation level. Adding 1 million triples takes ~5s in 2.7 and ~11s in 2.8.6 (SNAPSHOT_READ & READ_COMMITTED). 2.8.4 is slightly worse than 2.8.6 at ~13s. We've tested it with fewer triples and all other isolation levels as well.

My question is: is there anything we can do to get the read committed behaviour that 2.7 provided and maintain the same level of performance? This is preventing us from upgrading beyond 2.7.

Environment

Windows

Assignee

James Leigh

Reporter

Byron Moses

Labels

None

Affects versions

Priority

Major
Configure