Attention: We are retiring the ASP.NET Community Blogs. Learn more >

"use the filesystem as your database"

Stuart Langridge, writes about why he doesn't like databases and why to “use the filesystem as your database” (in context to the backend of a blog, I guess). IMO, it may hold some value to design a filesystem-driven data tier for ease-of-deployment or ease-of-maintenance (not that RDBMS administration these days is a highly complex affair for a sound DBA) or in the situation of “limited resource architecture” (where a physical database server may not be a feasible data source). But for ANY enterprise-level system or data-centric application, a database system undoubtedly provides high performance, efficiency, maximum storage capacity, extensive set of tools & components for interaction and management, a programming interface (i.e. SQL) and rapid integration capabilities (i.e. XML/SOAP) with legacy systems or other data sources. Persisting binary data (data blobs) in a database server is another story. Its a known fact that in a scenario where data query and server round-trips are high or constantly increasing, a filesystem-based data tier will bog down the entire application much quicker than a dedicated database-server. Talking of a larger picture, I doubt even Microsoft is moving away from this fact, with increased efforts to create a robust filesystem powered by a database engine (i.e. Longhorn WinFS). Filesystem vs database is an on-going discussion with two sides to a story. Considering SQLite, MySQL and PostgreSQL as inexpensive database server alternatives, I doubt cost has much to do with it.

[Quick Tip] Checkout OLAP Scribe - a utility for documenting your Analysis Services databases. It generates detailed information on the components you select. For example, it will tell you about the dimensions, measures, calculated members, source tables, last processed date, size, and read-only status of a selected cube. Definitely something you want to get & run periodically for snapshot documentation of your system. [via Christa Carpentiere]

2 Comments

  • "But for ANY enterprise-level system or data-centric application, a database system undoubtedly provides high performance, efficiency, maximum storage capacity, extensive set of tools & components for interaction and management, a programming interface (i.e. SQL) and rapid integration capabilities (i.e. XML/SOAP) with legacy systems or other data sources."



    I agree with some of that and disagree with some of that. "For any enterprise-level system...", absolutely. That's why I said, in my original post, "I am not a big fan of using 'real' databases...for tasks where you don’t have a lot of data." I'm not suggesting that Google should work by grepping text files here. Similarly, "a programming interface" and "extensive set of tools...for interaction and management" I agree with, *if you need them*. If you're doing complicated data queries, where you want to join tables (for example), then yes, use a database, definitely. But if you insert single records and retrieve single records, or recordsets containing only whole records, if all your SQL statements are basically "select * from mytable where criteria = whatever", then why use SQL? It's not buying you much there. In the same way, you don't need as many "interaction and management" tools for a filesystem as you do for a database because, well, they're simple. Setting up a scheduled backup? Just run "tar cvfpz /path/directory" in a cron job. Again, if you need the complexity that a database brings, then it's definitely a better solution. My rant was aimed at people incurring the overhead of a database (and more importantly making all their *users* incur the overhead of a database) when you really don't need one. I must confess that I fail to see how a database's XML layer can be a more rapid integration mechanism than a directory which is already full of XML files. Similarly, "maximum storage capacity" is pushing it a bit; a DB incurs a reasonable amount of extra storage just to store all the bits that make it a DB. Without compression, no database can take up less space than the data *in* it, and any compression that a DB can do the filesystem can do too.

    But, I have to stress the first point: you're talking about enterprise-class systems (where I agree with you) and I'm not. If everyone kept DBs for where they were required and didn't use the database sledgehammer to crack a lot of small nuts, there would be a lot less in the way of SQL injection errors and DB crashes in the world, I think.

  • Well you got a point. Filesystem does files. But did you just forgot that hierarchical fs is an archaic way of seeing things? What about metadata that NO os to date provide successfully? What about SORTING files, not just dumb-listing the content of a "folder"? Grepping is something but file management deserves better. Microsoft as on hell of a good idea. But the community as better free tools to circumvent the problem. Why not a better DB integration? Old-fashioned fs exists because of their os lying on top of it -- it doesn't stop us from improving file managment.



    simon

Comments have been disabled for this content.