Streams In JavaScript

Node.js is a way to move data from a source to a destination in bit-by-bit, to avoid any Out-Of-Memory Errors.

To understand what a stream does, consider a scenario with two buckets named source and destination. The source bucket is full of water, while the destination bucket is empty. We cannot move these two buckets from their spots, however, we have a third movable bucket called Buffer. We can use this buffer to transfer water from source to destination.

Node.js streams offer a powerful abstraction for managing the data flow in your applications. They excel at processing large datasets like videos or real-time transmissions without compromising performance.

This approach differs from traditional methods where the entire dataset is loaded into memory at once. Streams process data in chunks, significantly reducing memory usage.

Events-Driven Architecture

Node Js thrives on an event-driven architecture, making it ideal for real-time I/O. This means consuming input as soon as it's available and sending output as soon as the application generates it.

Streams seamlessly integrate with this approach, enabling continuous data processing.

They achieve this by emitting events at key stages. These events include signals for received data(data event) and the stream's completion(end event). Developers can listen to these events and execute custom logic accordingly. This event-driven nature makes streams highly efficient for real-time processing of data from external sources.

Streams Provide three key advantages over data-handling methods:-

1. Memory Efficiency

2. Improved Response Time

3. Scalability for real-time Processing

Types of Streams:-

1. Readable

2. Writable

3. Duplex

4. Transform

---

1.Readable Stream:

A Readable stream, reads data from a source and then feeds it to a pipeline bit by bit. For instance, let's create a read stream that can read from an array, and then pass it chunk by chunk.

Streams make use of event-emitters, which means they raise events, so we can listen for events.

Ex:- readStream.on('data',...)

readStream.on('end',...)

readStream.on(error',...)

const fs = require('fs');

const readStream = fs.createReadStream('./someVideo.Mov');

readStream.on('data', (chunk) => {
    console.log(`Size: ${chunk.length}`)
})

readStream.on('end', () => {
    console.log("Read stream Finished");
})

readStream.on('error', (error) => {
    console.log("Error Occurred");
    console.log(error);
})

The streams have two moods:

1. Flowing Stream Mode: It automatically pushes the chunk of data into the pipeline.

2. Non-Flowing Stream Mode: This means we have to ask for the data, and then the chunk will be pushed into the pipeline.

In the above code snippet, if we invoke, readStream.pause(), we will convert our read Stream (which was in flowing mode) to a non-flowing stream. We will have to ask for a chunk of data by using readStream.read(). In contrast, we can convert any non-flowing stream to a flowing stream by invoking resume() on the readstream, like shown in the below snippet.

const fs = require('fs');

const readStream = fs.createReadStream('./someVideo.Mov');

readStream.on('data', (chunk) => {
    console.log(`Size: ${chunk.length}`)
})

readStream.on('end', () => {
    console.log("Read stream Finished");
})

readStream.on('error', (error) => {
    console.log("Error Occurred");
    console.log(error);
})

readStream.pause();

process.stdin.on('data', (chunk) => {
    if(chunk.toString().trim() === 'finish') {
        readStream.resume();
    }
    readStream.read();
})

2.Writable Stream:

Writable Streams are like the counter-parts of readable streams. Just like how we read data in chunks, we can also write data in chunks. Writable Streams represent a destination for the incoming data. Writable stream enables us to read data from a readable source and do something with that data . Also, just like the readableStreams, writableStreams are also implemented in numerous places.

3.Duplex Stream:

A Duplex stream is a stream which implements both readable and writable. Readable stream will pipe data into the stream and the duplex stream can also write that data. Duplex streams represent the middle section of the pipelines, meaning, a duplex stream can be piped in between a readable and a writable stream.

A typical duplex stream doesn't change anything about the data. There's a type of duplex stream called TransformStreams. Using duplex streams, we can convert, streams into pipelines. Duplex streams are a necessary component to create complex pipelines. For instance, let us say we need to send data slowly, so we can create a throttle stream using duplexStreams and create a pipeline with it.

4.Transform Stream:

Transform streams are a special kind of duplex streams. Instead of simply passing the data from the read stream to write stream, transform streams change the data. In the below code snippet we have created a transform stream to take whatever input we give in the console and return an output where vowels have been replaced with Xs.

const { Transform } = require('stream');

class UppercaseTransform extends Transform {
    _transform(chunk, encoding, callback) {
        this.push(chunk.toString().toUpperCase());
        callback();
    }
}
const uppercase = new UppercaseTransform();

process.stdin.pipe(uppercase).pipe(process.stdout);

console.log('Type something and press eneter:');