Is VITA addition or replacement for Entity Framework

Jul 15, 2014 at 9:17 AM
Hi!

I read the developer documentation and VITA seems very interesting to me.
Hoewver, I still do not understand
*) whether VITA is an additional "module" that can be placed on top of Entity Framework datacontexts,
*) or whether VITA is a fully fledged ORM and as such a replacement for EF?

So what is VITA vs. Entity Framework?
Jul 15, 2014 at 3:32 PM
Vita is full replacement, done right, from scratch
Jul 15, 2014 at 5:30 PM
Hmmm... so to be honest - this is disappointing.

I thought VITA was another module that could bring some benefits to our existing technology stack.
However, "yet another ORM" is not what we want.

Why we like EF:
  • powerfull tool support in VS
  • support for Web API and Data Services
  • direct support for other frameworks like JayData and Breeze.js
  • (IMPORTANT) product support by microsoft
  • (IMPORTANT) code first migrations
I do not see all these benefits in VITA then, so what made you decide to not fork EF and extend it, but create a full fledged ORM yourself?
Is the EF codebase so terribly bad that it cannot be augmented towards VITA?
Jul 15, 2014 at 5:53 PM
Why did I decided to build a full-fledged framework instead of 'improving' EF? Good question. Because I was desperate. I was tired of waiting for something usable in big, enterprise scale apps. With number of tables in the database in scale of 500 or more. It's not that EF is not perfect, but can be 'improved' by adding some code here and there. The problem is that it is unusable from the start.
First, some illuminating points about EF. In one of LinkedIn forums, a developer asked is it OK to use EF for enterprise app. I advised against it, and when asked why by many, here's what I answered:
https://www.linkedin.com/groups/Is-it-ok-use-Entity-1465727.S.181783013
(scroll down to find response by Roman Ivantsov)
Here's an extract:

Here we go. Reliance on generated code - and generated code is really a bloated monster. Pitiful performance. Database schema update solution is a joke - watched ScottG struggle with it during presentation with just 2 tables; he gave up in the end - a proof that the whole idea of these Up/DownGrades is completely flawed. Unpredictable SQL generation - a simple LINQ statement may end up in 5K lines of SQL code, so you can never fully trust it. Huge effort spent on completely irrelevant things like OOP-style table inheritance - which contributed significantly to code bloat and internal complexity. Caching - very limited and insufficient for enterprise app. No multiple database support. Enums - it took them 5 years to finally implement it! Tells much about internal complexity(bloat?) and therefore system reliability. Continued denial of use of stored procedures for data access - which is still preferred way for most DBAs. No scaling for bigger models - confirmed by many. EF works sort of OK for small models, up to 100 tables, but completely melts down as you pass 150/200 entities. Does not matter whether you have one model or split it in many - multiple models have troubles of their own. Development by multiple developers - good luck with that; imagine 10 devs editing one EDM - and propagating changes. And propagating database schema updates. I know how difficult it is in real world team development: most shops have really sophisticated systems for this (custom built or third-party solutions). EF is definitely not up to this challenge.
With 500 tables enterprise app, pretty sure you'll have authorization requirements - to have granular permissions for every record, every field against the current user, his roles and "his data". Good luck building this on top of EF. All enterprise systems I know had to build custom ORMs if only for this reason - to have built-in authorization inside lower data management layer. I had seen systems brought down to a halt by authorization layer (not on EF, direct ADO.NET, but with EF the troubles will be bigger).
And as far as I know, inside MS not a single team is using EF, at least on anything important and complex enough (more than 10 tables). Except of course examples and sample projects for EF distribution.
Don't get me wrong - it might work for a one-shot projects - generate code, tweak it, go live and move on.
But enterprise app is a never ending evolution, over years and years, thru changing requirements, different front-ends, services, with new and new features. EF is unfit for this kind of process. The bad thing is that most of these troubles will manifest themselves when you are too far down the road, when you have the app grown to hundreds of tables, and have huge amount of code written.
(end of quote)
Jul 15, 2014 at 6:20 PM
The biggest problem with EF is that it is good enough for shiny quick demos at presentation speeches, but completely melts down when it comes to real-world apps. The tooling support you mention - yeah, they need these tools to manage the complexity and mess of their models, but unfortunately the designer simply stops working when come over 200 tables. Tested by many. I did not use EF in this scenario, it melted on me in much smaller cases, but what I saw was enough. I know people (not one person, many) who started excited with EF, and then came to a complete stop point, with no way out, and with huge pain had to abandon it and start from scratch.
Migrations - believe me, the story is not as shiny as MS paints it. Modifying DB model (whatever you use, Db-first or code-first) is a huge pain, requires hours of work to add even the simplest changes - a few columns or god forbid a new relationship. I know people who done it for months, before giving up completely. So think twice before you commit to this crap.
I have 20+ years of experience in big apps development, and I actually do not need to actually try a big project using a fwk to decide if it's good or not. There are many signs that actually show that people who built it had no idea about real enterprise apps, never built one, never supported one.
Now about your 'good' list about EF.
Powerful tool support for EF in VS. This is a big question how powerful and how reliable it is - apparently, it is not. Yes, VITA does not have any fancy designer, but the point is - it does not need one! All it needs is interface definitions - compact, readable and powerful enough. In our company, we used to generate table/entity diagrams from VITA entities (and from Db tables in management studio) - it is easy enough. But we stopped doing this simply because nobody needed used them. Looking at interface definitions tells the whole story. Nuff said.
Support for Web Api and data services. VITA supports Web Api in the same way as EF does - it provides a data access layer under Web Api. I provide a small example of building a RESTful web service with Web Api and VITA. OData is a different question.
Direct support for Breeze and JayData. What really is a point here is OData, not that EF supports JS frameworks directly. VITA does not support OData so far. I do not rule out it will be supported in the future, but keep in mind that OData is in fact MS proprietary protocol, no widely supported by the industry. You mention two JS frameworks that directly work with OData - but just 2 out of thousands there is not a big number. It does not really show big support. The whole idea of exposing IQueryable is questionable for me, and not only for me - google it. Note that many JS client frameworks do well without OData, and there are big apps built without it.
Jul 15, 2014 at 6:29 PM
Direct support from MS - do you realize that with thing like EF you might get 'this scenario is not supported' most of the time as SUPPORT from MS? The main point is what the product really supports out of the box, how to use it without calling MS. And there will be so many things that you gonna need and that are not supported...
Code first migrations... After reading this:
http://msdn.microsoft.com/en-us/data/jj591621.aspx
right, that's really a nice scenario, adding a single field. Again, watched ScottG himself trying simple migration at a demo, 2 years ago. Epic fail. Nuff said. VITA does automatic migrations, it check the current Db model and adjusts it (adds/removes columns, tables, relationships, indexes, etc), whenever it encounters differences with the model specified in code. No migrations, up/down version, just direct comparison and adjustment. The trouble with EF migrations (and that's what happened during this presentation) is that if something goes wrong in the middle of migration, you are screwed. Your database is left in the middle, between old version and new version, and you have to investigate and fix things manually. This never happens with VITA. Even if it fails for some reason in the middle, the next attempt will pick up where previous run stopped, and will try to adjust DbModel again.
Jul 15, 2014 at 6:55 PM
So VITA is a full-blown ORM that can be used inside existing technology stack. It does not require you to do Web stack in any custom way. Use WCF, Web Api or whatever you want.
Now, some critical points. When evaluating ORM solution, there are a few important requirements that are a MUST and absolutely critical in most modern apps (not all of course, but biz apps for sure). If ORM like EF does not have it, it is not acceptable.
There are several things that VITA has that are absolutely critical, but EF has nothing to offer in this area.
  1. Authorization. VITA allows completely transparent implementation of authorization checks at record/row level. If you're building an app that holds information that must be protected, and you must make sure that users see only what they own - this is a MUST requirement. And if it is not supported by the framework you use, you have to build it yourself, on top of it. And guess what - it turns your so nice and compact data access code into a mess. Every call to data, any attempt to modify it will surrounded by access-rights check. And believe me, this gonna be huge trouble. Messy, bloated code, SLOOOOOW and most importantly, unreliable as for completeness of security checks. And almost impossible to test properly. It is very, very hard to do authorization right - trust me. ALL ERP and business apps vendors implement their own, custom data access frameworks, with built-in authorization - for this main reason, no standard ORM provides it, so they have to do it from scratch. No ORM - except VITA. Read Authorization guide in documentation to see how it is implemented. Full-blown, TRANSPARENT, configurable at startup, authorization framework, with minimal perf penalty.
  2. Caching - all of the ORM, including EF, provide very PRIMITIVE cache solution. The biggest trouble is that you have to explicitly call cache in your code to check if data is cached, and if not - go to the database. EF is no better. Extremely primitive. And you can query single items by primary key only. VITA on the other hand, provides built-in, transparent cache - you can load whole tables into cache and then VITA will automatically rewrite any queries, including LINQ and execute them in cache, without going to database. App code does not have to be involved. Everything is configured at startup - which tables to cache and how.
  3. Component-based development - VITA is the first framework to offer ability to build application from modules/components. Several modules for critical functionality (like error log, web call log, transaction log) come out of the box. You can build your application from separate modules and merge them into one application - all necessary tables appear auto-magically and the whole thing works as if it was built as one app from the beginning. This is absolutely critical for big apps. Try figuring out how to do this with EF - good luck! Using multiple entity models is a big trouble, even in one app - confirmed by many.
Jul 15, 2014 at 7:01 PM
Again, not a single team inside MS is using EF. Kinda tells a lot. Something to seriously think about. Just last week, had a talk at a party with dev from one of MS teams (Azure tooling) - they tried EF initially, with around 300 tables, cursed it constantly and finally abandoned it for pure ADO.NET code. Nuff said.
So, do you want a framework that MS supports but does not use?
Jul 15, 2014 at 7:28 PM
WOW - that's quite a detailed response! :)

You should definetly copy-paste it and link it to "Entity Framework done right" at VITAs start page - that might help guys out there not being "frightened" to use some completely unknown ORM.

Sorry for me still asking, but I' just interested:
Why did you abandon NHibernate and not contribute to it? Does it have the same flaws?
OData:
Ok, I'll jut try VITA with Breeze.js and a Web API controller to see if I get it to work. Or could you give a brief list of things that would have to be done in order to get this to work?

Migrations:
So VITA supports kind-a migrations? I'll look into that.
last question - I promise
do you think PCL support would be possible?
Jul 15, 2014 at 8:01 PM
thanks!
Migrations - go see it in action, just add a few properties or even new entities to books model in samples, indexes(!) and see how it all appears in the database.
PCL - portable class libraries. Definitely yes/maybe. I'm a bit confused about what they are currently, aren't .NET IL assemblies supposed to be JIT-ted into binaries for target platform and therefore be already portable? As far as I know, Mono runs .NET assemblies as-is, most of the time. I will definitely look at providing Mono-usable solution, just no time for this right now. There's so much in todo list. One of the things is to remove some Win32-reliant functions (cache sync between domains using Windows signaling - will replace it with cache status table in Db)
NHibernate - downloaded and tried it. Did you see its size?! this one is one big stop signal. It is a big bloated monster, like most Java frameworks and their clones. As once was said, it's not an open source, it's an open mess. I could not find my way around it (just like with EF sources, just browse these, it is unbelievable pile of... ). At the time, NHibernate used HQL, queries as strings in specialized language. This as no-no for me. It must be LINQ, nothing else, so editor/compiler can check the names and types. Now I heard there are some attempts at LINQ, but it does not matter anymore. Another thing - db schema definitions/migrations. They had some support for this, but not good enough. Finally, there were inherent problems with internals on NHibernate. On example - holding strong references to all loaded records (I mentioned this somewhere) - if you load 50K records one-by-one, system runs out of memory. VITA uses weak references. Adjusting for this require deep changes in internals, and with such a huge codebase - I didn't think it would be possible.
Jul 15, 2014 at 8:01 PM
What's ahead in VITA - providing support for automatic locking/concurrency handling when updating complex documents, support for update/insert/delete operations using Linq, support for custom stored procs using Linq, support for explicit trans control, connection resiliency, direct SQL queries, Full-text search queries, etc, etc. Lots of work, but important thing - the way VITA is architected all these things are possible and doable, in a robust, reliable and easy-to-use way, declaratively and transparently. There's a clear road ahead.
Feb 1, 2015 at 5:33 PM
Hi Rivantsov, it seems that I'have been through some of your experiences with different ORM specially EF, I abandoned EF in previous versions for almost the same reasons as yours and after a a real project (although successful one) the list of my dislikes only grew with more concrete reasons! another experience with NHibernate proved that most of these beasts share the same fundamental problems (maybe NHibernate solves a few in theory and adds a few more like obsolete / incompatible plugins,..) I have mostly used some abstract/general custom and light-weight modules (or Dabber), but many times some interesting ideas of these frameworks and the problems they are there to solve is so tempting that I was about to research more and probably start my own light-weight framework to solve just those important problems and be flexible enough to fit any scenario without imposing many different design principles and way of doing things. Now I found your interesting work here and am wondering maybe you have done just that. I'm gonna give it a try, perhaps I can bring some value to it or help you out in the process extending / improving it. I have just a question, what do you think is the best way of getting familiar with your source code and do you have anything like a road map or feature list to give me an idea of where you are going?
Feb 2, 2015 at 7:02 AM
Well, welcome on board!
Try it, play, twist, look inside and out. Hope you'd appreciate how the 'business logic' comes out on top of the framework (BookStore sample) - compact, and expressive, without littering of any lower-level ADO/SQL crap. Also notice the reusability aspect (modules) - standard modules like Login and logging are ready-to-use in any app. In fact I use in real world apps. And integration with Web Api. What is missing now is a UI app sample to show how it might work together - this might be coming soon, I'm working with other dev on this.
The best way to explore it - go through code samples, there are a lot of them. Then go inside and see how it is done, how paths are altered for different servers and different 'situations'. Warning - it's not pretty in many places out there, but I'm fixing it, we'll get to better state.
Road ahead - see previous post. This is just a short list of things that are first-order priority, to make it basic-feature-set complete. Then possibilities are limitless, in different directions/dimensions. Are you good at UI? I'm very fascinated with Angular JS SPA architecture, I think this is a way to go, with JSON data api on server (based on VITA). What I would really love to have is better tooling. Like now I have command-line vdbtool for doing things like db-first code generation, and for producing diff SQL scripts for updating the database. It would be great to have a UI tool, similar to Management Studio, working for different DB servers.
I think I found the right way to fix many of the nasty problems we have, we are not there yet, there is a long way to go, but we'll get there. We are definitely far ahead of EF and other 'competing' wanna be's.
So play with it, and let me know if you have any questions or want to know anything about future things. Currently I'm in a rush to complete some big refactoring that will result in better overall structure and allow things like multi-tenant apps.

Roman
PS tomorrow going to meetup in Redmond, a guy from EF team will present EF7. They're still doing it. Will try to keep quiet and not giggle too loud -)
Feb 2, 2015 at 7:33 AM
and I would definitely welcome any help!
Apr 28, 2015 at 6:39 AM
Well, after playing a little with it - wha can I say, I liked it! :D

If I got some time I will try to help you. One good option is to put the project on github.

Did the framework support "Components" or "inheritance"?
Apr 28, 2015 at 7:15 AM
Welcome!
About components - not sure what you're talking about, but component development is done through EntityModules - the app is assembled from independently designed modules. Table/inheritance pattern is not supported
Apr 28, 2015 at 7:16 AM
about github - why? is there a strong reason?