Apr 11, 2016

What is stream processing?

How is stream processing any different from other methods of handling data?
Basically, instead of processing data in batches it is processed in real time, as it "streams" in.

"Stream processing....refers to a method of continuous computation that happens as data is flowing through the system. There are no compulsory time limitations in stream processing. For example, a system that simply output the count of words present in a Tweet for 99.9% of the tweets it encountered but output the complete works of Shakespeare for the remaining 0.1% of tweets is a valid stream processing system. There is no fixed time deadline on the output of the system when an input is received: the data is processed as it comes in and sometimes data might be awaiting processing. The only constraint on such a stream processing system is that its long term output rate should be faster or at least equal to the long term data input rate (otherwise the storage requirements of the system grow without bound)."
- Divye Kapoor on Quora. https://www.quora.com/Whats-the-difference-between-real-time-processing-and-stream-processing
Answer this