Performance benchmarks?

Apr 30, 2015 at 4:53 AM
Hi Roman,

Do you have any performance benchmarks? I am considering your framework for a new project and would be interested to know how it stacks up against micro-ORMs like Dapper, Massive, PetaPoco and OrmLite.

Thanks,
-geoff
Apr 30, 2015 at 8:12 AM
Edited Apr 30, 2015 at 5:40 PM
No, I don't have this ready to show, simply was never a priority for me to compete on this factor. I think as for ORM comparison, the performance should be just good enough, not terribly worse than other alternatives, and not much slower than no-ORM approach - hand-coded ADO.NET plus all surrounding entity creation/tracking code. Just yes/no flag, and if it is yes (performance acceptable), you bring other criteria for choosing the right ORM - which are more important in most cases.
Note that VITA is full-featured ORM so fair comparison should be with other similar ORMs like NH or EF, not Dapper.
However, having said that, I do believe that VITA's perf is quite good, even compared with micro (light-weight) ORMs. I constantly analyse log files and look at timing of data operations, and try to optimize internal code. There are several features in VITA that give it advantage in perf race:
  1. Batched execution of updates - if you update multiple records, all updates in a transaction (including Begin/Commit trans) are bundled into one SQL and executed in one roundtrip to the server. This might not show up in artificial benchmarks, but in real-world apps that edit a lot of data it is a big booster,
  2. Built-in cache, including full-set cache when whole tables are loaded in memory and all data requests (including LINQ queries) are satisfied from cache.
  3. Automatic LINQ query caching, so a query is translated to SQL once. Linq2Sql can do this, but you have to explicitly use CompiledQuery object. I believe the same in EF
  4. Connection reuse mode (connection is not closed but is held open for the entire web request) - getting connection from pool is not free, it requires a roundtrip to server to 'reset' environment
and some other stuff, big and small.
I did look at some benchmarks; there was ORM Battle, not supported anymore (for last 3 years), but I could not make it run on my machine. There is another one here:
https://github.com/FransBouma/RawDataAccessBencher.
It covers a mix of light/full featured ORMs, and I am not sure how representative is the benchmark compared to real life apps. This I was able to run after a few tweaks, but did not get to adding a version for VITA - just too much other urgent stuff on the plate. Wanna try - if you're concerned about perf?
In my experience, VITA was used already on a number of projects, mid-size databases in the cloud, and perf was never an issue. I mean, perf of Fwk, there are always perf troubles, but they all were about search queries against big tables, and these were 'fundamental' problems, even hand crafted query did not do better, so it was not ORM issue.
To sum it up, I strongly believe the perf will not be your issue; if anything it will not be VITA, but inefficient code in upper levels (biz logic) like repeated queries for the same stuff - in my experience this is the source of perf troubles, not underlying low-level interface to DB
Roman