What I don’t know much about is how to improve query efficiency. From my understanding, you will throw a lot of information, which will help make the query more efficient, just turn your structured database into a bunch of keys and values?
Have I missed this completely?
What you need to ask is, does my intended use case make sense to switch?
You missed a little bit. The key is that you sometimes don’t have an index (just like you use a general relational database). Even if you have an index, the ability to tie them together is difficult, how does a relational database perform. NoSQL solutions have many novel structures that make many uses simple and easy to use. For example, Redis is a data structure-oriented database, which is very suitable for quickly building any content with a queue or its pub-sub structure. MongoDB is a free-format document database that stores documents as JSON (BSON) and can be developed quickly. The structure of the BigTable solution is not so simple, but it extends the idea of one row, that is, a series of columns-the key-value pairs contained in each row are efficiently arranged on disk. You can use techniques such as ElasticSearch to build an inverted index.
Not everything needs the consistency guarantee or disk layout of traditional RDBMS. Another major use case of NoSQL is huge scalability, many solutions (such as BigTable-HBase / Cassandra) are designed to be easily graded and scaled (SQL is not easy)! Cassandra is specifically designed to be used without SPOF. In addition, column-oriented data storage optimizes disk speed (and reduces write-amplification) through sequential reads. That being said, unless you really need it, a traditional SQL server is usually good enough.
There are advantages and disadvantages. Personally, I use a mixture of the two. Use the right tools for the right work, which may cause PostgreSQL or MySQL to be more frequent.
You can use a basic key-value system to create a SQL table with two columns, a unique key and a value. This is quite fast. You don’t need to do any relationship or association or collation of the data. Just find the value and return. This is an over-simplification, NoSQL database does have many interesting functions and applications, beyond simple K, V storage.
I don’t know whether your scientific data is suitable for most NoSQL implementations, it depends on the data. If you look at HBase or Cassandra, it may suit the needs of scientists (use the appropriate rowkey design-the timestamp cannot be viewed in OpenTSDB first). I know that many companies store the readings in the daily fat row and the sensor readings in Cassandra by using random sequential separators and the UUID of the sensor. Every day, new databases are created around specific use cases so that the answers may change. For specific use cases, you can get huge rewards for using specific data storage at the cost of flexibility and tools.
I have suggested that I investigate the Key/Value data system to replace the relational database I have been using.
What I don’t know much about is how to improve query efficiency. From my understanding, you will throw a lot of information, which will help make the query more efficient, just turn your structured database into a bunch of keys and values?
Have I missed this completely?
The key advantage of relational databases is the ability to associate and index information. Most “NoSQL” systems do not provide relational algebra or a good query language.
What you need to ask is, does my intended use case make sense to switch?
You missed a little bit. The key is that you sometimes don’t have an index (just like you use a general relational database). Even if you have an index, the ability to tie them together is difficult, how does a relational database perform. NoSQL solutions have many novel structures that make many uses simple and easy to use. For example, Redis is a data structure-oriented database, which is very suitable for quickly building any content with a queue or its pub-sub structure. MongoDB is a free-format document database that stores documents as JSON (BSON) and can be developed quickly. The structure of the BigTable solution is not so simple, but it extends the idea of one row, that is, a series of columns-the key-value pairs contained in each row are efficiently arranged on disk. You can use techniques such as ElasticSearch to build an inverted index.
Not everything needs the consistency guarantee or disk layout of traditional RDBMS. Another major use case of NoSQL is huge scalability, many solutions (such as BigTable-HBase / Cassandra) are designed to be easily graded and scaled (SQL is not easy)! Cassandra is specifically designed to be used without SPOF. In addition, column-oriented data storage optimizes disk speed (and reduces write-amplification) through sequential reads. That being said, unless you really need it, a traditional SQL server is usually good enough.
There are advantages and disadvantages. Personally, I use a mixture of the two. Use the right tools for the right work, which may cause PostgreSQL or MySQL to be more frequent.
You can use a basic key-value system to create a SQL table with two columns, a unique key and a value. This is quite fast. You don’t need to do any relationship or association or collation of the data. Just find the value and return. This is an over-simplification, NoSQL database does have many interesting functions and applications, beyond simple K, V storage.
I don’t know whether your scientific data is suitable for most NoSQL implementations, it depends on the data. If you look at HBase or Cassandra, it may suit the needs of scientists (use the appropriate rowkey design-the timestamp cannot be viewed in OpenTSDB first). I know that many companies store the readings in the daily fat row and the sensor readings in Cassandra by using a random sequence separator and the UUID of the sensor. Every day, new databases are created around specific use cases so that the answers may change. For specific use cases, you can get huge rewards for using specific data storage at the cost of flexibility and tools.