If you try to use 10 gig for any form of business or web processing, you may need to install one that can keep up on the front end Load splitter for 10GB traffic.
ps, I must clarify that NIDS handles 1:1 traffic on the machine that sees the traffic-that is, in the worst case you handle the traffic on the same machine Every byte; and business/network processing is 1: many: many machines and an order of magnitude of many bytes to process.
– edit –
Now you have mentioned data There is a gap between transmissions (no matter what the standard 10Gb network version cannot keep up with 10Gb), we need to know what is being processed in order to make suggestions.
– Edit 2 –
When you use multiple threads, Berkeley DB (a database with a simple data model) behaves like an enterprise database (in terms of transaction rate). If you want to write to disk at a high rate, you should explore this solution. Raid settings may be needed to increase throughput-in terms of IO throughput and protection, raid 0 1 is the best.
I have two debian boxes connected via CX4 cables Between two 10 GbE cards. One is to generate data quickly (between 4Gbits/s and 16Gbits/s), and the other needs to be able to fetch all of them and store them in RAM for later parsing. I am this low-level encoding Newbie, and would be happy to accept any ideas about using a wide range of methods (Do I need DMA? RDMA?), or tips and tricks that may be applicable. Thank you!
If you want to continuously process 1 GB of traffic per second, you need a very wide bus and very fast processing speed. My experience comes from NIDS. You need special The hardware to perform NIDS consistently to process 100MB (1 Gigabit Ethernet) data (10 Gb is another universe). Ram will not help you because you can fill in GB in 5-10 seconds, and 1 GB can provide A large number of requests.
If you are trying to use 10 gig for any form of business or web processing, you may need to install a load splitter that can keep up with 10GB of traffic on the front end.
< p>ps, I must clarify that NIDS processes 1:1 traffic on the machine that sees the traffic-that is, in the worst case you process every byte on the same machine; while business/network processing is 1 : Many: Many machines and an order of magnitude of many bytes to process.
– Edit –
Now you have mentioned that there is a gap between data transfers (regardless of the standard 10Gb network Version can’t keep up with 10Gb), we need to know what is being processed in order to make suggestions.
– Edit 2 –
When you use multiple threads, Berkeley DB (with A database with a simple data model) behaves similarly to an enterprise database (in terms of transaction rate). If you want to write to disk at a high rate, you should explore this solution. You may need raid settings to increase throughput-in terms of IO throughput In terms of quantity and protection, raid 0 1 is the best.