On Oct 30, 2009, at 1:31 PM, Dan Linder wrote:
What I'm looking for is some general comments and discussion about the mental task of mapping these hash tables into a SQL table. I'm not really looking for a tool, more a high level discussion about ways to store the data and still remain flexible.
An RDBMS is the most flexible thing around. Ignore your current implementation details and focus on the problems that your data addresses. Organize your data by business concept (table) that would make sense to current and future employees of that business. Keep whatever performance metrics are important to you in mind as you design the tables.
Write software to map your hashes into that model. Push all the data into an RDBMS. Benchmark your most common interactions with that data. Optimize your indexes for those use cases. Benchmark again. Prep all your tools for your new model. QA. Migrate the data. svn rm the old stuff. :)
HTH, j