The Art of I/O (1st Session of 4)
A magical introduction to input and output signals
Pre-Requisites
To begin, you need
Course Overview
In this course, we will learn the core forms of Javascript I/O magic:
- functions
- closures
- modules
- callbacks
- events
- streams
By becoming proficient with the above, you'll have no trouble altering space and time... cyber space and time.
Hello World!
To say hello, write a file hello-world.js with the following:
// hello-world.js
console.log('hello world!');
Run the file with node hello-world.js.
Function Magic
Functions accept input, return output, and maybe perform effects.
By "abstracting out" lower details, functions allow us to think at a higher level.
// add-tax.js
// `addTax` is a function
// that accepts a number `x`
function addTax (x) {
// and returns `x` multiplied by `1.15`.
return x * 1.15;
}
addTax(100); // 115
Functional Hello
Great, now we let's abstract our hello into a function.
Let's write a new file hello-world-function.js:
// hello-world-function.js
function hello () {
console.log('hello world!');
}
hello();
See our function run with node hello-world-function.js.
Musical Functions
Closure Magic
Closures are a useful pattern of functions that "close over variables".
// multiply-by.js ---------------------------
// `multiplyBy` is a function
// that accepts a number `n`
function multiplyBy (n) {
// and returns a function that
// accepts a number `x`
// and returns `x` multiplied by `n`.
return function (x) {
return x * n;
};
};
var double = multiplyBy(2);
var triple = multiplyBy(3);
var addTax = multiplyBy(1.15);
double(100); // 200
triple(100); // 300
addTax(100); // 115
Say
Sweet, what if we wanted to speak with closures?
Time for hello-bye.js!
// hello-bye.js
function say (text) {
return function () {
console.log(text);
};
}
var hello = say("hello world!");
var bye = say("bye world...");
hello();
bye();
Weeee! node hello-bye.js.
Module Magic
Modules are chunks of code that be used in other chunks of code.
Be lazy! Try to write as little new code as possible and instead re-use as much existing code as possible.
Require Magic
If we have a file named my-favorite-color.js:
// my-favorite-color.js
var myFavoriteColor = "green"
module.exports = myFavoriteColor;
Then in another file in the same directory we can "require" the first file with:
var myFavoriteColor = require("./my-favorite-color");
console.log(myFavoriteColor); // green
The 'say' function would make a great require-able module. :)
// say.js
function say (text) {
return function () {
console.log(text);
};
}
module.exports = say;
// hello-bye-modular.js
var say = require('./say');
var hello = say("hello world!");
var bye = say("bye world...");
hello();
bye();
The moment of truth, node hello-bye-modular.js.
But we can go deeper into modularity. :)
// hello.js
var say = require('./say');
module.exports = say("hello world!");
// bye.js
var say = require('./say');
module.exports = say("bye world...");
// hello-bye-more-modular.js
var hello = require('./hello');
var bye = require('./bye');
hello();
bye();
Tee hee hee.. node hello-bye-more-modular.js
Async Magic (1/3)
For example, when you're at a restaurant, if you order food they will immediately take your order and then make you wait around until the food is ready. In the meantime they can take other orders and start cooking food for other people. Imagine if you had to wait at the register for your food, blocking all other people from ordering while they cooked your food! This is called "blocking" I/O because all I/O (food orders) happens one at a time. Javascript, on the other hand, is "non-blocking", which means it can handle many orders at once.
Async Magic (2/3)
Everything we've done so far is "synchronous": tasks are handled sequentially, the interpreter waits to execute the next task until the current task (like reading from the filesystem) is done.
Async Magic (3/3)
The power of Javascript I/O lies in being "asynchronous": tasks are handled simultaneously, the interpreter never waits and instead queues up tasks to respond when each current task is done.
Callback Magic
When you order at a restaurant, the arguments of your request (what you order) are written down with an associated callback (what to do when your order is ready).
Callbacks are functions that are executed "async", or at a later time. As with any function, they can be stored in variables and passed around with different names.
Sync vs Async
// hello-bye-more-modular.js
var hello = require('./hello');
var bye = require('./bye');
hello(); // say 'hello'
bye(); // say 'bye'
// hello-bye-async.js
var timers = require('timers'); // a built-in module
var hello = require('./hello');
var bye = require('./bye');
timers.setTimeout(hello, 0); // say 'hello' immediately after
bye(); // say 'bye'
Hello and Delayed Bye
What if we wanted to say 'hello' and delay our 'bye' for a bit?
// hello-delayed-bye.js
var timers = require('timers'); // a built-in module
var hello = require('./hello');
var bye = require('./bye');
timers.setTimeout(hello, 0); // say 'hello' immediately after
// delay time (in milliseconds)
// before saying 'bye'
timers.setTimeout(bye, 5 * 1000);
Hello......... bye. node hello-delayed-bye.js
Repeated Hello
Actually, it's sad to say 'bye'. Let's keep saying 'hello'!
// hello-on-repeat.js
var timers = require('timers');
var hello = require('./hello');
// repeat interval (in milliseconds)
// to keep saying 'hello'
timers.setInterval(hello, 1 * 1000);
Hello. Hello. Hello... node hello-on-repeat.js
Say With An Argument
Saying 'hello' is boring now, what if we could specify what to say when we run our code?
// say-arg.js
var timers = require('timers');
var say = require('./say');
// get speech from command-line arguments
var speech = process.argv[2];
// set repeating interval (in milliseconds) to say speech
timers.setInterval(say(speech), 1 * 1000);
Meow. node say-arg.js 'rawrrrrr! :3'
fs Magic
fs is a built-in module used for reading from and writing to the filesystem.
// read-me.js
var fs = require('fs');
var filename = "./read-me.js";
fs.readFile(filename, afterReadFile);
function afterReadFile (err, result) {
// if error, let's throw it
if (err) { throw err; }
// otherwise, do something with result
console.log(result.toString());
}
Event Magic
Javascript I/O uses "events", or when things happen. Timeouts, intervals, connections, clicks, key presses, etc are all events. When an async function is run, the associated callback listens for when the async function is done, and handles the event.
Callback Convention
Most callbacks in Javascript I/O have a conventional form, they take two arguments: err and result. (Which you can name to whatever you like when defining your callback functions).
Async Magic
// read-more.js
var fs = require('fs');
var filenames = [
"./read-me.js", "./read-more.js",
"./hello.js", "./bye.js", "./say.js",
];
function afterReadFile (name) {
return function (err, result) {
if (err) { throw err; }
console.log(name);
};
}
filenames.forEach(function (filename) {
fs.readFile(filename, afterReadFile(filename));
});
Advanced Callback Magic
The top-to-bottom order that you declare callbacks doesn't matter, only the logical/hierarchical nesting of them. First split code up into functions, then use callbacks to declare if one function depends on another function being done.
Go Forth and Magic
More Advanced Magic
go on if you dare.
the rest of the slides did not fit within the presentation, but they are still available as notes.
Writing a file
So now that we can say what we want, let's write what we say into a journal.txt file.
// append-journal-txt.js -------------------------------------------
var fs = require('fs');
// get entry from command-line arguments
var entry = process.argv[2];
// append entry to journal, handling any errors
appendJournal(entry, defaultErrorHandler);
function appendJournal (entry, callback) {
// append entry to journal.txt file and pass along callback
fs.appendFile("./journal.txt", entry, callback);
}
function defaultErrorHandler (err) {
if (err) { throw err; } // nothing better to do
}
Write with node append-journal-txt.js 'today i learned i/o' and read with cat journal.txt
Writing a directory
Having a single journal file for all entries is no good, let's name our entries and store them as separate files in a directory.
// write-journal.js ---------------------------------------------------
var fs = require('fs');
var defaultErrorHandler = require('./default-error-handler')
// get name and text from command-line arguments
var name = process.argv[2];
var entry = process.argv[3];
// append entry to journal, handling any errors
appendJournal(name, entry, defaultErrorHandler);
function appendJournal (name, entry, callback) {
// append entry to respective journal file and pass along callback
fs.appendFile("./journal/" + name, entry, callback);
}
Create a journals directory with mkdir journal, write an entry with node write-journal.js io 'today i learned i/o', list entries with ls journal, and read entries with cat journal/*
Listing our journal entries
Now that we have a directory of our journal entries, let's list their names.
// list-journal.js -----------------------------------------------
var fs = require('fs');
var defaultErrorHandler = require('./default-error-handler');
getEntryNames(function (err, entryNames));
if (err) { return defaultErrorHandler(err); }
// for each entry name, log it to console
entryNames.forEach(console.log);
});
function getEntryNames (callback) {
// get entry names from journal directory and pass callback
fs.readdir("./journal", callback);
}
List your journal entries with node list-journal.js.
Count length of journals
Sweet, we have the names of the journal entries, now let's count how long the entries are (by number of characters).
// count-journal.js
//-----------------------------------------------------------------------
var fs = require('fs');
var defaultErrorHandler = require('./default-error-handler');
var getEntryNames = require('./get-entry-names');
// get entry names
getEntryNames(function (err, entryNames));
if (err) { return defaultErrorHandler(err); }
// for each entry name,
entryNames.forEach(function (entryName) {
// count length of entry from journal
countEntry(entryName, function (err, count) {
if (err) { return defaultErrorHandler; }
// print name and length of entry
console.log(entryName, ":", count);
});
});
function countEntry (entryName, callback) {
fs.readFile(entryName, function (err, entry) {
if (err) { return callback(err); } // pass error to callback
callback(null, entry.toString().length);
});
}
Count your journal entries with node count-journal.js.
Async Javascript, or Javascript that uses callbacks, is hard to get right intuitively. A lot of code ends up looking messy, with nested callbacks indenting on forever. If you notice this, here's a guide to help. :)
Speak Art
Making sense is overrated, let's speak art.
// speak-art.js
//----------------------------------------------------------
function randomNumberInRange (min, max) {
return Math.floor(Math.random() * (max - min) + min);
}
function randomItemInArray (array) {
return array[randomNumberInRange(0, array.length)];
}
// words inspired by http://youtu.be/7Ye4facLqlM
var words = [
"blue", "ribbon", "yarn", "pidgeon", "dawn",
"double", "take", "\"yawn\"", "never", "sleep", "eat",
"habit", "hygiene", "hydrate", "shitty", "wine", "art",
"starcraft", "ramen", "night", "or", "morning",
"coffee", "code", "fish", "and", "chips",
];
function randomWord () {
return randomItemInArray(words);
}
function speak () {
var words = [randomWord(), randomWord(), randomWord()];
console.log(words.join(" "));
}
setInterval(speak, 1 * 1000); // time in milliseconds
Meow Meow Meow. node speak-art.js
Speak Art With Modules
Hmm, maybe someone else already wrote randomNumberInRange.
Yup, npm install random-number-in-range.
// speak-art-1.js
//--------------------------------------------------------------
var randomNumberInRange = require('random-number-in-range');
function randomItemInArray (array) {
return array[randomNumberInRange(0, array.length)];
}
// words inspired by http://youtu.be/7Ye4facLqlM
var words = [
"blue", "ribbon", "yarn", "pidgeon", "dawn",
"double", "take", "\"yawn\"", "never", "sleep", "eat",
"habit", "hygiene", "hydrate", "shitty", "wine", "art",
"starcraft", "ramen", "night", "or", "morning",
"coffee", "code", "fish", "and", "chips",
];
function randomWord () {
return randomItemInArray(words);
}
function speak () {
var words = [randomWord(), randomWord(), randomWord()];
console.log(words.join(" "));
}
setInterval(speak, 1 * 1000); // time in milliseconds
oewMay. node speak-art-1.js
Speak Art With Modules
Hmm, maybe someone else already wrote randomItemInArray.
Yup, npm install random-item-in-array.
// speak-art-2.js
//--------------------------------------------------------------
var randomItemInArray = require('random-item-in-array');
// words inspired by http://youtu.be/7Ye4facLqlM
var words = [
"blue", "ribbon", "yarn", "pidgeon", "dawn",
"double", "take", "\"yawn\"", "never", "sleep", "eat",
"habit", "hygiene", "hydrate", "shitty", "wine", "art",
"starcraft", "ramen", "night", "or", "morning",
"coffee", "code", "fish", "and", "chips",
];
function randomWord () {
return randomItemInArray(words);
}
function speak () {
var words = [randomWord(), randomWord(), randomWord()];
console.log(words.join(" "));
}
setInterval(speak, 1 * 1000); // time in milliseconds
ReowMawr. node speak-art-2.js
Speak Random Words
Hmm, maybe someone else already wrote a random word generator.
Yup, npm install random-word.
While we're at it, let's npm install repeat-function to clean our code even more.
// speak-random.js
//-----------------------------------------------------------
var randomWord = require('random-word');
var repeatFunction = require('repeat-function');
function speak () {
var words = repeatFunction(3, randomWord);
console.log(words.join(" "));
}
setInterval(speak, 1 * 1000); // time in milliseconds
Easy Easy Easy! node speak-random.js
Write random words
Now that we have a way of generating random words, how about we write 100 sentences of 3 words each into a "journal.txt". :)
We'll be using the built-in fs module to write to a file.
// write-random-journal.js
//------------------------------------------------------
var fs = require('fs');
var defaultErrorHandler = require('./default-error-handler');
var randomWord = require('random-word');
var repeatFunction = require('repeat-function');
function randomSentence () {
var words = repeatFunction(3, randomWord);
return words.join(" ") + ".";
}
var sentences = repeatFunction(100, randomSentence);
fs.writeFile("journal.txt", sentences.join(" "), defaultErrorHandler);
Run node write-random-journal.js to write, run cat journal.txt to read.
Write more random words
We need more words! How about we write 10 journals of 1 to 10,000 sentences of 3 to 17 words.. :)
// write-random-journals.js
//----------------------------------------------------------------
var fs = require('fs');
var randomNumberInRange = require('random-number-in-range');
var randomWord = require('random-word');
var repeatFunction = require('repeat-function');
function randomJournal () {
var numSentences = randomNumberInRange(0, 11);
var sentences = repeatFunction(numSentences, randomSentence);
return sentences.join(" ");
}
function randomSentence () {
var numWords = randomNumberInRange(3, 18);
var words = repeatFunction(numWords, randomWord);
return words.join(" ") + ".";
}
var journals = repeatFunction(10, randomJournal);
journals.forEach(function (journal) {
var name = "journals/" + randomWord() + ".txt";
fs.writeFileSync(name, journal);
});
Start by creating a new directory to hold our journals with mkdir journals. Then run node write-random-journals.js to write, and run ls -lh journals to see a list of what we've made.
Read Our Journal List
Okay okay okay, enough writing. Let's read our journals, starting with a simple list of their names.
// read-journal-names.js
var fs = require('fs');
// read journal names from filesystem
var journalNames = fs.readdirSync("journals");
// for each journal name, log it in the console
journalNames.forEach(console.log);
node read-journal-names.js should look similar to ls journals
Read Our Journals
Now that we have a list of our journal names, let's read the words! Although instead of filling our console with words, let's count how many words are in each journal and print that.
// read-journal-lengths.js
var fs = require('fs');
// read journal names from filesystem
var journalNames = fs.readdirSync("journals");
// for each journal name,
journalNames.forEach(function (name) {
// read journal from filesystem
var journal = fs.readFileSync(name, { encoding: 'utf8' });
// split journal into words
var words = journal.split(/[\.\s]+/);
// log journal name and number of words
console.log(name, words.length);
});
Echo Server
// echo-server.js
var http = require('http'); // built-in module
function handler (req, res) {
req.pipe(res);
}
var server = http.createServer(handler);
server.listen(5000);
Start with node server-echo.js.
Echo Client
// echo-client.js
var http = require('http'); // built-in module
var path = "http://localhost/";
function handler (res) {
var req = http.get(path, handler);
res.pipe(req);
}
http.get(path, handler);
Static Server
// static.js
var http = require('http'); // built-in module
function handler (req, res) {
fs.readFile("." + req.url, function (err, contents) {
res.write(contents);
res.end();
});
}
var server = http.createServer(handler);
server.listen(5000);