September 5, 2014 Topics

A daily roundup: Google Drive admin FAQ, a custom Mongoose connection manager, and a new HR opening — September 5, 2014.

Express + Mongoose MongoDB Connection Configuration Sample Code

Sample code for connecting Express + Mongoose to MongoDB, walked through commit by commit.

[Express.js] How to Fix (node:10151) UnhandledPromiseRejectionWarning: TypeError: res.status is not a function

If Express.js throws `res.status is not a function`, check the order of `req`, `res` in your handler.

[Heroku] Ruby on Rails 4 + Mongoid + MongoHQ '500 Internal Server Error'

Fixing the Mongoid ':consistency is an unknown option' 500 error on Heroku by trimming mongoid.yml.

[KeystoneJS] How to Change the Blog Feature URL from /blog

How to change the KeystoneJS blog feature URL from /blog to /news via a set of source diffs.

How to Create a Node.js CMS 'KeystoneJS' App and Deploy it to Heroku

Build a KeystoneJS CMS app and deploy it to Heroku with MongoDB and other add-ons.

How to use moment.js and moment-timezone in MongoDB shell

Load moment.js and moment-timezone into the MongoDB shell — with a mongorc.js snippet for automatic loading.

mesh.js - extending MongoDB shell to use moment.js, underscore.js and other useful libraries

Extending the MongoDB shell with mesh.js so moment.js and underscore.js are available, with a date-arithmetic example.

[MongoDB] Display Count in mongo shell Can Be Set with DBQuery.shellBatchSize

Change the mongo shell page size with DBQuery.shellBatchSize to skip typing "it" over and over.

[MongoDB] How to Check Execution Time of aggregate

Measuring MongoDB aggregate execution time by wrapping mongo --eval with the shell's time command.

[MongoDB] Using aggregate to Sort by Array Length

Aggregate pipeline that counts array elements with $unwind + $group and sorts by size.

[MongoDB] Checking for Array Field Element Existence

Use `'arrayField.0': { $exists: true }` to find documents whose array field is not empty.

How to Configure MongoDB Atlas for Development Environment

Setting up MongoDB Atlas for development: free-tier region choice and Network Access whitelist notes.

[MongoDB] Failed global initialization: BadValue Invalid or no user locale set. Please ensure LANG and/or LC_* environment variables are set correctly.

Work around the mongo shell BadValue locale error by running `export LC_ALL=C`.

[MongoDB] Solution for exception: can't convert from BSON type EOO to Date

Guard $year/$month with $match and `$type: 9` to avoid the aggregate EOO to Date error.

[MongoDB] How to Fix "$err" : "Can't canonicalize query: BadValue cannot compare to undefined", "code" : 17287

Reproducing and guarding against the MongoDB code 17287 undefined-comparison error.

[MongoDB] Check the dbpath where database files are stored

Look up MongoDB's dbpath from `db.serverCmdLineOpts()` in the mongo shell.

[MongoDB] How to Check the Storage Engine

Check the MongoDB storage engine via `db.serverStatus().storageEngine`.

[MongoDB] How to use db.runCommand( { cloneCollection } )

Clone a collection between mongod hosts with a filtered `cloneCollection` runCommand.

[MongoDB] Query to Get Documents with nth Decimal Places and Update with mth Decimal Place Values

Use $where and `toFixed(2)` to round prices with 3+ decimal places down to two.

MongoDB document size can be obtained with Object.bsonsize()

Measure the byte size of a MongoDB document with `Object.bsonsize()` in the mongo shell.

[MongoDB] Difference Between Code 11000 and 11001 in E11000 duplicate key error index

Reproducing why MongoDB 2.4.5 returned both 11000 and 11001 for duplicate key error index, and confirming 3.0.2 unified them to 11000.

[MongoDB] How to execute queries from Terminal with arguments

Passing arguments into a MongoDB query file from the Terminal using the mongo shell's --eval option.

[MongoDB] Outputting field values containing line breaks in a format that can be imported by Google Spreadsheets

Exporting MongoDB fields with line breaks as CSV that Google Sheets can import out of the box.

[MongoDB] How to output query results to standard output and save to file

Piping MongoDB query results straight into a file with mongo --quiet and shell redirection.

[MongoDB] Error: failed to connect to [localhost:27017]

How a missing localhost entry in /etc/hosts broke a Node.js MongoDB connection on 27017.

[MongoDB] Field Naming Conventions

Since the MongoDB manual mixes snake_case and lowerCamelCase, standardize on whichever is easier for your application code.

[MongoDB] Query to find documents with specific ObjectId from all collections

A mongo shell snippet that scans every collection to locate a document when all you have is its ObjectId.

[MongoDB] Display all records with find().forEach(printjson) or .toArray()

Two ways around mongo shell's 'has more' cutoff: .forEach(printjson) and .toArray().

Using MongoDB's findOneAndUpdate with Replica Set Awareness to Reduce Load on Primary DB

Prefixing findOneAndUpdate with a secondaryPreferred findOne to offload reads from the MongoDB primary.

[MongoDB] How to Resolve Error: getMore: cursor didn't exist on server, possible restart or timeout?

Tuning indexes first, then falling back to the noTimeout cursor option to fix MongoDB's getMore cursor error.

GUI Client for MongoDB

Installing Robomongo as a GUI client for MongoDB in a development environment.

[MongoDB] Index names can only be set up to a maximum of 127 bytes

Reproducing MongoDB's 127-byte index name limit with successful and failing createIndex calls.

[Mac] Installing MongoDB with Homebrew

Installing MongoDB with Homebrew on macOS and setting it up as a launchd auto-start service.

[MongoDB] Getting year-month-day string from ISODate

Extracting YYYY-MM-DD from a MongoDB ISODate with toISOString().substring(0, 10).

[MongoDB] Date Formatting in JavaScript

A mongo shell JavaScript snippet that formats a Date as YYYY/MM/DD HH:MM:SS.

Created .jshintrc Specialized for MongoDB Queries

A MongoDB-specific .jshintrc that registers mongo shell methods as globals to silence JSHint noise.

[MongoDB] How to Change LogLevel (Log Level) Settings

How to switch MongoDB logLevel between 0, 1, and 5 with setParameter.

How to Measure Total Execution Time of MongoDB Queries

Measure total execution time of a multi-line MongoDB query block with a simple new Date() diff.

[MongoDB] Connecting to Another DB from mongo shell to Execute Queries

Open a second connection with connect() and reach another MongoDB server from the same shell.

[MongoDB] Backup with mongodump and Restore with mongorestore

mongodump / mongorestore commands for backing up and restoring a whole DB or a single collection.

[MongoDB] Search Conditions for Deeply Nested Fields

Why dot notation matches nested MongoDB fields but object-form conditions require an exact match.

[MongoDB] MongoError: Runner error: Overflow sort stage buffered data usage of x bytes exceeds internal limit of 33554432 bytes

Adding an index on the sort field resolved the Overflow sort stage error in Express + Mongoose.

[MongoDB] $pull and $pullAll for Removing Specific Elements from Array Fields

$pull and $pullAll examples for removing single or multiple elements from a MongoDB Array field.

[MongoDB] Query to Get Documents Where a Specific Text Field Has x or More Characters

Use $ne with $where to fetch MongoDB documents whose text field length meets a minimum.

[MongoDB] How to rename/change database names

Rename a MongoDB database by copyDatabase to the new name, then dropDatabase the old one.

[MongoDB] How to start two instances on different ports (Mac / Homebrew)

Run two Homebrew MongoDB instances side by side on ports 27017 and 27018 with a second config.

Use printjson() to output Objects in MongoDB shell

Use printjson() instead of print() to inspect Object values inside the MongoDB shell.

[MongoDB] How to resolve SyntaxError: Unexpected end of input, Unexpected token }

When a regex literal breaks the MongoDB shell with SyntaxError, use the RegExp constructor instead.

[MongoDB] How to Fix "sort stage buffered data usage of x bytes exceeds internal limit of 33554432 bytes" Error

Adding a Mongoose schema index on the sort field cleared the sort stage buffered data limit error.

Steps to build specific version MongoDB server [Docker Edition]

Run a specific MongoDB version locally with docker run, then verify from the mongo CLI.

When copying and pasting MongoDB query results with tab-delimited strings, tab characters may disappear, so file output is best

Tab characters can disappear when pasting Mongo query output from Terminal — writing to a file and importing fixes it.

[MongoDB] Calculate unique values using distinct and aggregate

Sample MongoDB queries for computing unique values with distinct, plus an aggregate fallback for the 16MB cap.

[MongoDB] Command to Remove Fields

How to remove fields in MongoDB with $unset, with samples for single-document and bulk updates.

[Mongoose] Implementation Strategy for Adding Default Field Definitions Midway

Adding a default to a Mongoose Schema later fills in existing data with that default too — here's how to handle it.

[MongoDB] Bank Account Schema Definition Sample Code

Sample field names for a Mongoose bank-account Schema — bank_name, branch_code, account_holder_name_kana, and more.

[Mongoose] How to Specify Two or More Different OR Conditions

To combine two or more different OR conditions in Mongoose, wrap the $or clauses inside an explicit $and.

[Mongoose] Using Only createdAt with timestamp options

How to keep only createdAt in Mongoose by implementing it yourself in pre save/update hooks.

[Mongoose] Check if Specific Fields Have Changed with isDirectModified, isModified, modifiedPaths

How Mongoose's isDirectModified, isModified, and modifiedPaths differ when detecting field changes on a document.

[Mongoose] When lean option is enabled, a plain javascript object is returned instead of MongooseDocument

Mongoose's lean option returns plain JS objects, so save, virtuals, and instance methods stop working.

Converting Mongoose documents to JSON and returning response

Use Mongoose's lean() to return documents as a JSON response from Express with res.json.

[Mongoose] Use mongoose.Types.ObjectId.isValid for ObjectId Validation

Validate ObjectId strings in Mongoose with mongoose.Types.ObjectId.isValid.

[Mongoose] How to fix RangeError: attempt to write outside buffer bounds

The RangeError: attempt to write outside buffer bounds in Mongoose comes from BSON's 16MB per-document limit.

[Mongoose] Setting validation runValidators: true in update() and findOneAndUpdate()

Turn on validators for Mongoose update() and findOneAndUpdate() with the runValidators: true option.

[Mongoose] Saving Error Objects to Schema.Types.Mixed Fields

Convert Error objects with toString() or lodash.toPlainObject() before saving them to a Mongoose Schema.Types.Mixed field.

Explanation of mongoose.Schema type: ObjectId

What type: ObjectId in a mongoose.Schema means, including MongoDB _id and Mongoose ref.

[Mongoose] Schema.Types.Mixed Requires .markModified Before save

If a Schema.Types.Mixed value won't save, you need to call markModified before save.

[Node.js] Implementing Simple Auto Increment with Mongoose

A simple Mongoose auto increment using findOne + sort to add 1 to the current maximum code.

[Mongoose] You Can Skip Validation with model.save({ validateBeforeSave: false })

From Mongoose 4.2.0 you can skip validation with save({ validateBeforeSave: false }).

[Mongoose] When doing batch processing with stream, setting noCursorTimeout: true option might make you happy

A long-running Mongoose stream batch that stops mid-way is fixed by passing noCursorTimeout: true.

[Mongoose] Trouble with Stream Processing Only Default batchSize When Sorting by Unindexed Field

A Mongoose stream sorted by an unindexed field stops at 1000 docs — cause and workarounds.

[Mongoose][timekeeper] TypeError: Undefined type `FakeDate` at `fieldName`

Calling new mongoose.Schema inside timekeeper.freeze throws a FakeDate TypeError — cause and fix.

[Mongoose] TypeError: Cannot read property 'options' of undefined

An object-typed value in a type: ObjectId field triggers the options undefined TypeError — here's why.

[Mongoose] How to Fix Error: Unable to invalidate a subdocument that has not been added to an array.

The 'unable to invalidate a subdocument' Mongoose error explained via a bad type: ObjectId value.

[Mongoose] How to Fix (node:42286) DeprecationWarning: current URL string parser is deprecated, and will be removed in a future version. To use the new parser, pass option { useNewUrlParser: true } to MongoClient.connect.

Fix the MongoDB URL parser DeprecationWarning by passing useNewUrlParser: true to mongoose.connect.

[Mongoose] How to Fix (node:42424) DeprecationWarning: current Server Discovery and Monitoring engine is deprecated, and will be removed in a future version. To use the new Server Discover and Monitoring engine, pass option { useUnifiedTopology: true } to the MongoClient constructor.

Fix the Server Discovery and Monitoring DeprecationWarning with useUnifiedTopology: true.

[Node.js] Stubbing Mongoose virtual attributes with sinon

Stub Mongoose virtual attributes from tests with sinon.stub plus stub.withArgs.

[Node.js] Let's use stable version to avoid MONGOOSE WARNING

Fixing the !!! MONGOOSE WARNING !!! banner in a Node.js app by pinning package.json to a stable Mongoose version.

Lessons from 'Open Source: From Community to Commercialization | Andreessen Horowitz'

Lessons from a16z on OSS business models and why community — not code — is the real competitive moat.

Ruby on Rails4 + Mongoid + RSpec

Bootstrapping a Rails 4 app with Mongoid and RSpec, including DatabaseCleaner wiring.

[Shell Script] Getting formatted dates for yesterday, today, and tomorrow

Using `date -d '1 days ago'` and friends to get yesterday, today, and tomorrow formatted as `YYYY-MM-DD`.

Open Source Software (OSS) and Company List Featured by Unusual Ventures

A summary of OSS-related articles on Unusual Ventures and the featured OSS/company list.