Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Major] Add server-side input validation #182

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
3009e3c
[Models] Delete sanitizeQuery function from SanitizationAndValidation.ts
dchege711 Jun 16, 2024
ae312b1
[InAppRouter.test] First crack
dchege711 Jun 24, 2024
cb2d14f
[InAppRouter.test] tRPC supported testing
dchege711 Jun 24, 2024
80120b4
[InAppRouter.test] Restore the supertest-based tests
dchege711 Jun 24, 2024
88c417b
[Deps] npm install --save express-mongo-sanitize
dchege711 Jun 24, 2024
67a13ad
[server] Use mongoSanitize middleware
dchege711 Jun 24, 2024
5f11fd9
[Utils] Optionally log in populateDummyAccountWithCards
dchege711 Jun 25, 2024
01cee17
[fixtures] Add public user and register dummy account before each test
dchege711 Jun 25, 2024
2fb1fba
[InAppRouter.test] Better tests: ensure valid ID and presence of avai…
dchege711 Jun 25, 2024
6a0bbfc
[Deps] npm install --save zod
dchege711 Jun 29, 2024
f09e369
[DB] Parse fetchPublicCard input
dchege711 Jun 29, 2024
f561587
[Deps] npm i --save validator
dchege711 Jun 29, 2024
b45152c
[tRPC] Use isMongoId to validate CardID
dchege711 Jun 29, 2024
7c8a9a4
[InAppRouter.test] Add more cases for fetchPublicCard
dchege711 Jun 29, 2024
08e794f
[InAppRouter] Add validators for searchPublicCards and flagCard
dchege711 Jun 29, 2024
bdeece6
[InAppRouter] Add validation for fetchCard
dchege711 Jun 29, 2024
76b9d45
[Deps] npm install --save-dev chai-as-promised
dchege711 Jun 29, 2024
2801bbe
[Deps] npm i --save-dev @types/chai-as-promised
dchege711 Jun 29, 2024
e7fc6b7
[InAppRouter.test] Failed to use chaiAsPromised
dchege711 Jun 29, 2024
759bd19
Revert "[InAppRouter.test] Failed to use chaiAsPromised"
dchege711 Jun 29, 2024
83bd750
Revert "[Deps] npm i --save-dev @types/chai-as-promised"
dchege711 Jun 29, 2024
5917d1c
Revert "[Deps] npm install --save-dev chai-as-promised"
dchege711 Jun 29, 2024
b94879c
[InAppRouter] Add validation for addCard
dchege711 Jun 29, 2024
1ab4726
[InAppRouter] Add validation for updateCard
dchege711 Jun 29, 2024
6a7ba70
[InAppRouter] Add validation for trashCard
dchege711 Jun 29, 2024
01220bb
[InAppRouter] Add validation for deleteCard
dchege711 Jun 29, 2024
43a3c93
[InAppRouter] Add validation for searchCards
dchege711 Jun 29, 2024
729f4f5
[InAppRouter] Add validation for duplicateCard
dchege711 Jun 29, 2024
91450f7
[InAppRouter] Add validation for restoreCardFromTrash
dchege711 Jun 29, 2024
56080c5
[InAppRouter] Add validation for streak
dchege711 Jun 29, 2024
e2dd1f2
[InAppRouter] Add validation for settings
dchege711 Jun 29, 2024
9e0a9df
[InAppRouter.test] Rename tests to clarify that we're checking the pr…
dchege711 Jun 29, 2024
b1eb74d
[ES] Add test debugging command
dchege711 Jun 30, 2024
b244a78
[Auth] Fix logInBySessionToken path in logIn middleware
dchege711 Jun 30, 2024
046ff5a
[InAppRoutes.test] E2E for an Express-powered auth-guarded page
dchege711 Jun 30, 2024
b2e7dd7
[EmailClient] Mock the sendMail call to avoid pinging Mailgun
dchege711 Jun 30, 2024
a7b4aba
[tRPC] Remove unused settings endpoint
dchege711 Jun 30, 2024
a8f70f8
[InAppRouter.test] Add failing test cases for /account
dchege711 Jun 30, 2024
ffa0fba
[search-bar] Stop passing "Infinity" as that doesn't serialize to a n…
dchege711 Jun 30, 2024
9322738
[ES] Better typing for req.session
dchege711 Jun 30, 2024
b1cae4a
[server] Only use secure cookies in prod
dchege711 Jun 30, 2024
a261687
[EmailClient] Only send actual mail in prod
dchege711 Jun 30, 2024
338bb19
[EmailClient] Stop logging if suppressing emails
dchege711 Jun 30, 2024
f49952f
[InAppRoutes] Add validation middleware for /account POST
dchege711 Jun 30, 2024
5421cd2
[AuthRoutes] Add tests for / and /login and validation for /login
dchege711 Jun 30, 2024
97036a3
[AuthRoutes] Add validation for /register-user
dchege711 Jun 30, 2024
cbeb87f
[AuthRoutes] Add validation for /send-validation-email
dchege711 Jun 30, 2024
a56fbeb
[AuthRoutes] Add validation for /verify-account
dchege711 Jun 30, 2024
eab4cad
[AuthRoutes] Add validation for /reset-password
dchege711 Jun 30, 2024
e677bad
[AuthRoutes] Add validation for /reset-password-link
dchege711 Jun 30, 2024
c93bc1a
[InAppRouter.test] Rename tRPC tests to indicate they test endpoints
dchege711 Jun 30, 2024
97cbf78
[App] Uninstall express-mongo-sanitize and remove usage
dchege711 Jun 30, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .vscode/launch.json
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,12 @@
"name": "npm debug",
"request": "launch",
"type": "node-terminal"
},
{
"command": "npx ts-mocha --config ./.mocharc.yml --timeout 0",
"name": "test:server",
"request": "launch",
"type": "node-terminal"
}
]
}
12 changes: 6 additions & 6 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 3 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -78,8 +78,9 @@
"nodemailer": "^6.9.9",
"path": "0.12.7",
"sjcl": "1.0.7",
"validator": "^13.7.0",
"xss": "1.0.3"
"validator": "^13.12.0",
"xss": "1.0.3",
"zod": "^3.23.8"
},
"devDependencies": {
"@eslint/js": "^9.4.0",
Expand Down
3 changes: 2 additions & 1 deletion src/@types/augmentations.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,14 @@
*/

import type { NextFunction, Request, Response } from "express";
import type { Session } from "express-session";

import type { AuthenticateUser } from "../models/LogInUtilities.js";
import type { IUser } from "../models/mongoose_models/UserSchema.js";

declare module "express" {
interface Request {
session?: {
session?: Session & {
message?: string;
user?: AuthenticateUser;
};
Expand Down
4 changes: 2 additions & 2 deletions src/controllers/AuthenticationController.ts
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ function renderForm(
* using certain URLs
*/
export const requireLogIn: RequestHandler = (req, res, next) => {
if (!req.session?.user) {
if (!req.session?.user && !req.cookies.session_token) {
res.redirect(StatusCodes.SEE_OTHER, allPaths.LOGIN);
} else if (req.session?.user) {
if (req.body && req.body.userIDInApp) {
Expand All @@ -46,7 +46,7 @@ export const requireLogIn: RequestHandler = (req, res, next) => {
}
next();
} else if (req.cookies.session_token) {
logInBySessionToken(req.cookies.session_token, res, next);
logInBySessionToken(req, res, next);
}
};

Expand Down
64 changes: 62 additions & 2 deletions src/controllers/ControllerUtilities.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
import { unlink } from "fs";

import { Request, Response } from "express";
import { NextFunction, Request, Response } from "express";
import { rateLimit } from "express-rate-limit";

import { StatusCodes } from "http-status-codes";
import { z, ZodError } from "zod";

import * as config from "../config";
import { UserRecoverableError } from "../errors";
import { AuthenticateUser } from "../models/LogInUtilities";
Expand Down Expand Up @@ -110,3 +111,62 @@ export const rateLimiter = rateLimit({
standardHeaders: true, // Return rate limit info in the `RateLimit-*` headers
legacyHeaders: false, // Disable the `X-RateLimit-*` headers
});

type ZodSchemaTypes =
| ReturnType<typeof z.object>
| ReturnType<typeof z.string>
| z.ZodEffects<z.ZodObject<z.ZodRawShape>>
| z.ZodEffects<z.ZodString>;

/**
* Helper function for validating that `req.body` matches the schema provided.
* Adapted from [1].
*
* [1]: https://dev.to/osalumense/validating-request-data-in-expressjs-using-zod-a-comprehensive-guide-3a0j
*/
export function validationMiddleware(schema: ZodSchemaTypes) {
return (req: Request, res: Response, next: NextFunction) => {
try {
schema.parse(req.body);
next();
} catch (error) {
if (error instanceof ZodError) {
const errorMessages = error.errors.map((issue) => ({
message: `${issue.path.join(".")} is ${issue.message}`,
}));
res.status(StatusCodes.BAD_REQUEST).json({
error: "Invalid data",
details: errorMessages,
});
} else {
res.status(StatusCodes.INTERNAL_SERVER_ERROR).json({
error: "Internal Server Error",
});
}
}
};
}

// TODO: How to share code with `validationMiddleware` above?
export function pathValidationMiddleware(schema: ZodSchemaTypes) {
return (req: Request, res: Response, next: NextFunction) => {
try {
schema.parse(req.path);
next();
} catch (error) {
if (error instanceof ZodError) {
const errorMessages = error.errors.map((issue) => ({
message: `${issue.path.join(".")} is ${issue.message}`,
}));
res.status(StatusCodes.BAD_REQUEST).json({
error: "Invalid data",
details: errorMessages,
});
} else {
res.status(StatusCodes.INTERNAL_SERVER_ERROR).json({
error: "Internal Server Error",
});
}
}
};
}
18 changes: 15 additions & 3 deletions src/fixtures.ts
Original file line number Diff line number Diff line change
@@ -1,10 +1,16 @@
import { Runner } from "mocha";
import { fake, replace } from "sinon";

import { PORT } from "./config";
import { close as closeEmailClient } from "./models/EmailClient";
import { deleteAllAccounts } from "./models/LogInUtilities";
import { close as closeEmailClient, transporter } from "./models/EmailClient";
import {
deleteAllAccounts,
registerUserAndPassword,
} from "./models/LogInUtilities";
import { addPublicUser } from "./models/Miscellaneous";
import { closeMongooseConnection } from "./models/MongooseClient";
import { app } from "./server";
import { dummyAccountDetails } from "./tests/DummyAccountUtils";

/**
* Root hooks are ran before (or after) every test in every file.
Expand All @@ -16,7 +22,9 @@ export const mochaHooks = {
* In both parallel and serial modes, runs before each test.
*/
async beforeEach() {
await deleteAllAccounts([]);
return deleteAllAccounts([])
.then(() => addPublicUser())
.then(() => registerUserAndPassword(dummyAccountDetails));
},
};

Expand All @@ -35,6 +43,10 @@ export const mochaHooks = {
* Run once before all tests.
*/
export async function mochaGlobalSetup(this: Runner) {
// Mock out the outgoing email. There doesn't seem to be a way to mock out
// ES6 exports.
replace(transporter, "sendMail", fake.resolves("Mocked out sendMail."));

this.server = app.listen(PORT);
console.log(`Server listening on port ${PORT}`);
}
Expand Down
13 changes: 4 additions & 9 deletions src/models/CardsMongoDB.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
import { FilterQuery, SortOrder } from "mongoose";
import * as MetadataDB from "./MetadataMongoDB";
import { Card, ICard, ICardDocument } from "./mongoose_models/CardSchema";
import { sanitizeCard, sanitizeQuery } from "./SanitizationAndValidation";
import { sanitizeCard } from "./SanitizationAndValidation";

export type CreateCardParams = Pick<
ICard,
Expand Down Expand Up @@ -84,7 +84,6 @@ export function read(
projection =
"title description descriptionHTML tags urgency createdById isPublic",
): Promise<ICard | null> {
payload = sanitizeQuery(payload);
const query: FilterQuery<ICard> = { createdById: payload.userIDInApp };
if (payload.cardID) { query._id = payload.cardID; }
return Card.findOne(query).select(projection).exec();
Expand Down Expand Up @@ -179,7 +178,7 @@ export function search(
*/
return collectSearchResults(
computeInternalQueryFromClientQuery(
sanitizeQuery(payload),
payload,
{ createdById },
),
);
Expand Down Expand Up @@ -290,13 +289,13 @@ export function publicSearch(
): Promise<CardsSearchResult[]> {
return collectSearchResults(
computeInternalQueryFromClientQuery(
sanitizeQuery(payload),
payload,
{ isPublic: true },
),
);
}

export type ReadPublicCardParams = Omit<ReadCardParams, "userIDInApp">;
export type ReadPublicCardParams = Pick<ReadCardParams, "cardID">;

/**
* @description Read a card that has been set to 'public'
Expand All @@ -323,7 +322,6 @@ export function readPublicCard(
function _readPublicCard(
payload: ReadPublicCardParams,
): Promise<ICardDocument | null> {
payload = sanitizeQuery(payload);
if (payload.cardID === undefined) {
return Promise.reject("cardID is undefined");
}
Expand All @@ -350,7 +348,6 @@ export interface DuplicateCardParams {
export async function duplicateCard(
payload: DuplicateCardParams,
): Promise<ICard> {
payload = sanitizeQuery(payload);
const originalCard = await _readPublicCard({ cardID: payload.cardID });
if (originalCard === null) {
return Promise.reject("Card not found!");
Expand Down Expand Up @@ -395,7 +392,6 @@ export interface FlagCardParams {
* as its keys. If successful, the message will contain the saved card.
*/
export async function flagCard(payload: FlagCardParams): Promise<ICard> {
payload = sanitizeQuery(payload);
const flagsToUpdate: Partial<
Pick<ICard, "numTimesMarkedAsDuplicate" | "numTimesMarkedForReview">
> = {};
Expand Down Expand Up @@ -429,7 +425,6 @@ export type TagGroupings = string[][];
export function getTagGroupings(
payload: TagGroupingsParam,
): Promise<TagGroupings> {
payload = sanitizeQuery(payload);
return Card
.find({ createdById: payload.userIDInApp })
.select("tags").exec()
Expand Down
13 changes: 9 additions & 4 deletions src/models/EmailClient.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,18 +8,19 @@
* @module
*/

import { createTransport } from "nodemailer";
import { createTransport, SentMessageInfo } from "nodemailer";

import Mail = require("nodemailer/lib/mailer");

import {
APP_NAME,
EMAIL_ADDRESS,
IS_PROD,
MAILGUN_LOGIN,
MAILGUN_PASSWORD,
} from "../config";

const transporter = createTransport({
export const transporter = createTransport({
pool: true,
host: "smtp.mailgun.org",
port: 587,
Expand All @@ -38,9 +39,13 @@ const transporter = createTransport({
*
* @returns {Promise} takes a JSON with `success` and `message` as the keys
*/
export function sendEmail(mailOptions: Mail.Options): Promise<void> {
export function sendEmail(mailOptions: Mail.Options): Promise<SentMessageInfo> {
mailOptions.from = `${APP_NAME} <${EMAIL_ADDRESS}>`;
return transporter.sendMail(mailOptions).then(() => {});
if (IS_PROD) {
return transporter.sendMail(mailOptions);
} else {
return Promise.resolve({} as SentMessageInfo);
}
}

/**
Expand Down
3 changes: 0 additions & 3 deletions src/models/LogInUtilities.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@ import { Card } from "./mongoose_models/CardSchema";
import { Metadata } from "./mongoose_models/MetadataCardSchema";
import { IToken, Token } from "./mongoose_models/Token";
import { IUser, User } from "./mongoose_models/UserSchema";
import { sanitizeQuery } from "./SanitizationAndValidation";

const DIGITS = "0123456789";
const LOWER_CASE = "abcdefghijklmnopqrstuvwxyz";
Expand Down Expand Up @@ -260,8 +259,6 @@ export type RegisterUserAndPasswordParams =
export async function registerUserAndPassword(
payload: RegisterUserAndPasswordParams,
): Promise<string> {
payload = sanitizeQuery(payload);

const conflictingUser = await User.findOne({
$or: [{ username: payload.username }, { email: payload.email }],
}).exec();
Expand Down
Loading
Loading