User Service

CircleCI insights

dev

The Mentor building block enables effective mentoring interactions between mentors and mentees. The capability aims to create a transparent eco-system to learn, connect, solve, and share within communities. Mentor is an open-source mentoring application that facilitates peer learning and professional development by creating a community of mentors and mentees.

System Requirements

Operating System: Ubuntu 22
Node.js: v20
PostgreSQL: 16
Citus: 12.1
Apache Kafka: 3.5.0

Setup Options

Elevate user services can be set in local using two methods:

Dockerized service with local dependencies(Intermediate)

Expectation: Run single docker containerized service with existing local (in host) or remote dependencies.

Local Dependencies Steps

Update dependency (Mongo v4.1.4, Kafka etc) IP addresses in .env with "host.docker.internal".

Eg:
```
 #Kafka Host Server URL
 KAFKA_URL = host.docker.external:9092
```
Find host.docker.internal IP address and added it to mongod.conf file in host.

Eg: If host.docker.internal is 172.17.0.1, mongod.conf:
```
# network interfaces
net:
    port: 27017
    bindIp: "127.0.0.1,172.17.0.1"
```
Note: Steps to find host.docker.internal IP address & location of mongod.conf is operating system specific. Refer this for more information.

Build the docker image.

/ELEVATE/user$ docker build -t elevate/user:1.0 .

Run the docker container.

For Mac & Windows with docker v18.03+:

$ docker run --name user elevate/user:1.0

For Linux:

$ docker run --name user --add-host=host.docker.internal:host-gateway elevate/user:1.0`

Refer this for more information.

Remote Dependencies Steps

Update dependency (Mongo v4.1.4, Kafka etc) Ip addresses in .env with respective remote server IPs.

Eg:

 #DB Connectivity Url
 DATABASE_URL=postgres://postgres:postgres@localhost:5432/elevate-user

 #Kafka Host Server URL
 KAFKA_URL = 11.2.3.45:9092

Add Bind IP to mongod.conf in host:

Follow instructions given here.

Note: Instructions might differ based on MongoDB version and operating system.

Build the docker image.

/ELEVATE/user$ docker build -t elevate/user:1.0 .

Run the docker container.

$ docker run --name user elevate/user:1.0

Local Service with local dependencies(Hardest)

Expectation: Run single service with existing local dependencies in host (Non-Docker Implementation).

Installations

Install Node.js LTS

Refer to the NodeSource distributions installation scripts for Node.js installation.

$ curl -fsSL https://deb.nodesource.com/setup_lts.x | sudo -E bash - &&\
sudo apt-get install -y nodejs

Install Build Essential

$ sudo apt-get install build-essential

Install Kafka

Refer to Kafka Ubuntu 22.04 setup guide

Install OpenJDK 11:
```
$ sudo apt install openjdk-11-jdk
```

Download and extract Kafka:

$ sudo wget https://downloads.apache.org/kafka/3.5.0/kafka_2.12-3.5.0.tgz
$ sudo tar xzf kafka_2.12-3.5.0.tgz
$ sudo mv kafka_2.12-3.5.0 /opt/kafka

Configure Zookeeper:

$ sudo nano /etc/systemd/system/zookeeper.service

Paste the following lines into the zookeeper.service file:

/etc/systemd/system/zookeeper.service
[Unit]
Description=Apache Zookeeper service
Documentation=http://zookeeper.apache.org
Requires=network.target remote-fs.target
After=network.target remote-fs.target

[Service]
Type=simple
ExecStart=/opt/kafka/bin/zookeeper-server-start.sh /opt/kafka/config/zookeeper.properties
ExecStop=/opt/kafka/bin/zookeeper-server-stop.sh
Restart=on-abnormal

[Install]
WantedBy=multi-user.target

Save and exit.

Reload systemd:
```
$ sudo systemctl daemon-reload
```

Configure Kafka:

$ sudo nano /etc/systemd/system/kafka.service

Paste the following lines into the kafka.service file:

[Unit]
Description=Apache Kafka Service
Documentation=http://kafka.apache.org/documentation.html
Requires=zookeeper.service

[Service]
Type=simple
Environment="JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64"
ExecStart=/opt/kafka/bin/kafka-server-start.sh /opt/kafka/config/server.properties
ExecStop=/opt/kafka/bin/kafka-server-stop.sh

[Install]
WantedBy=multi-user.target

Save and exit.

Reload systemd:
```
$ sudo systemctl daemon-reload
```
Start Zookeeper:
```
$ sudo systemctl start zookeeper
```
Check status:
```
$ sudo systemctl status zookeeper
```
Zookeeper service status should be shown as active (running).
Start Kafka:
```
$ sudo systemctl start kafka
```
Check status:
```
$ sudo systemctl status kafka
```
Kafka status should be shown as active (running).

Install Redis

Refer to Redis Ubuntu 22.04 setup guide

Update the package list:
```
$ sudo apt update
```
Install Redis:
```
$ sudo apt install redis-server
```

Configure Redis for systemd:

$ sudo nano /etc/redis/redis.conf

Find the supervised directive and change it to "systemd" as follows:

. . .
# If you run Redis from upstart or systemd, Redis can interact with your
# supervision tree. Options:
#   supervised no      - no supervision interaction
#   supervised upstart - signal upstart by putting Redis into SIGSTOP mode
#   supervised systemd - signal systemd by writing READY=1 to $NOTIFY_SOCKET
#   supervised auto    - detect upstart or systemd method based on
#                        UPSTART_JOB or NOTIFY_SOCKET environment variables
# Note: these supervision methods only signal "process is ready."
#       They do not enable continuous liveness pings back to your supervisor.
supervised systemd
. . .

Save and exit.

Restart the Redis service:
```
$ sudo systemctl restart redis.service
```

Install Single-Node Citus (Distributed Database)

Refer to official Citus single-node setup

Download and install Citus:

$ curl https://install.citusdata.com/community/deb.sh | sudo bash
$ sudo apt-get -y install postgresql-16-citus-12.1

Switch to the PostgreSQL user:
```
$ sudo su - postgres
```
Set the PostgreSQL bin directory in the PATH and create a directory for Citus:
```
$ export PATH=$PATH:/usr/lib/postgresql/16/bin
$ cd ~
$ mkdir citus
```
Initialize the Citus database:
```
$ initdb -D citus
```

Configure Citus in citus/postgresql.conf:

$ echo "shared_preload_libraries = 'citus'" >> citus/postgresql.conf

Start the Citus server:

$ pg_ctl -D citus -o "-p 9700" -l citus_logfile start

Create the Citus extension:

$ psql -p 9700 -c "CREATE EXTENSION citus;"

Check the Citus version:

$ psql -p 9700 -c "select citus_version();"

You should see an output similar to the following, indicating that Citus is successfully installed:

postgres=# select citus_version();
                                       citus_version
----------------------------------------------------------------------------------------------------
 Citus 12.1.1 on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0, 64-bit
(1 row)

Install PM2

Refer to How To Set Up a Node.js Application for Production on Ubuntu 22.04.

Exit the postgres user account and run the following command

$ sudo npm install pm2@latest -g

Setting up Repository

Clone the user repository to /opt/backend directory

opt/backend$ git clone -b develop-2.5 --single-branch "https://github.com/ELEVATE-Project/user.git"

Install Npm packages from src directory

backend/user/src$ sudo npm i

Create .env file in src directory

user/src$ sudo nano .env

Copy-paste the following env variables to the .env file:

NB : Make sure to update the credentials according to your configurations.

ACCESS_TOKEN_EXPIRY= 10
ACCESS_TOKEN_SECRET= asadsd8as7df9as8df987asdf
ADMIN_INVITEE_UPLOAD_EMAIL_TEMPLATE_CODE= invitee_upload_status
ADMIN_SECRET_CODE= Na7ad23ws5cm3kfmw24dmdsflaksd
API_DOC_URL=/user/api-doc
APPLICATION_ENV=development
APPLICATION_PORT=3001
APP_NAME=MentorED

AWS_ACCESS_KEY_ID= "adsfg98a7sdfg"
AWS_BUCKET_ENDPOINT="s3.ap-south-1.amazonaws.com"
AWS_BUCKET_REGION="ap-south-1"
AWS_SECRET_ACCESS_KEY="asd9786fg9a8sd/asdfg9a8sd7fg"


AZURE_ACCOUNT_KEY=asd897gfa09sd87f09as8d
AZURE_ACCOUNT_NAME=mentoring
CLEAR_INTERNAL_CACHE=userinternal
CLOUD_STORAGE= GCP
DEFAULT_AWS_BUCKET_NAME=mentoring-dev-storage
DEFAULT_AZURE_CONTAINER_NAME=mentoring-images
DEFAULT_GCP_BUCKET_NAME=mentoring-dev-storage

DEFAULT_ORGANISATION_CODE= default_code
DEFAULT_ORG_ID= 1
DEFAULT_QUEUE= user-queue
DEFAULT_ROLE= mentee
DEV_DATABASE_URL= postgres://shikshalokam:slpassword123@localhost:9700/elevate_user
DISABLE_LOG= false
EMAIL_ID_ENCRYPTION_ALGORITHM= aes-256-cbc
EMAIL_ID_ENCRYPTION_IV= a19f1ewaqwei9e03edkc32e
EMAIL_ID_ENCRYPTION_KEY= 9bszawjkckw2e3dm35fcw27ws4ed5rftg6y6y7y7654tf4rwq5tr0ol2qa9owsie
ENABLE_EMAIL_OTP_VERIFICATION=true
ENABLE_LOG=true
ERROR_LOG_LEVEL=silly
EVENT_ENABLE_ORG_EVENTS=true
EVENT_ORG_LISTENER_URLS=http://localhost:3567/mentoring/v1/organization/eventListener
GCP_PATH=gcp.json
GCP_PROJECT_ID=sl-dev-project
GENERIC_INVITATION_EMAIL_TEMPLATE_CODE=generic_invite
INTERNAL_ACCESS_TOKEN= Fqdkfaswekdlwe
INTERNAL_CACHE_EXP_TIME= 86400
INVITEE_EMAIL_TEMPLATE_CODE= invite_user
IV= LKYTTAqkajswiawqw/Z==
KAFKA_GROUP_ID=dev.users
KAFKA_TOPIC= dev.topic
KAFKA_URL= localhost:9092
KEY= W/m2cr/aMswjrdsa23sgfy5e34d+bKcbAWZSLjJP2qY=
MENTEE_INVITATION_EMAIL_TEMPLATE_CODE= invite_mentee
MENTORING_SERVICE_URL= http://localhost:3000
MENTOR_INVITATION_EMAIL_TEMPLATE_CODE= invite_mentor
MENTOR_REQUEST_ACCEPTED_EMAIL_TEMPLATE_CODE= mentor_request_accepted
MENTOR_REQUEST_REJECTED_EMAIL_TEMPLATE_CODE= mentor_request_rejected
MENTOR_SECRET_CODE=4567
NOTIFICATION_KAFKA_TOPIC=dev.notification
ORG_ADMIN_INVITATION_EMAIL_TEMPLATE_CODE= invite_org_admin
OTP_EMAIL_TEMPLATE_CODE= emailotp
OTP_EXP_TIME= 86400
PORTAL_URL= "https://dev.elevate-mentoring.shikshalokam.org/auth/login"
RATING_KAFKA_TOPIC= dev.mentor_rating
REDIS_HOST= redis://localhost:6379
REFRESH_TOKEN_EXPIRY= 183
REFRESH_TOKEN_SECRET=371hkjadidy2ashiKAkajshdkid23iuekw71yekiaskdvkvegxvy23t78veQwexqviveit6ttZyeeytx62tx236uv
REFRESH_VIEW_INTERVAL=30000
REGISTRATION_EMAIL_TEMPLATE_CODE= registration
REGISTRATION_OTP_EMAIL_TEMPLATE_CODE= registrationotp
SALT_ROUNDS= 10
SAMPLE_CSV_FILE_PATH= sample/bulk_user_creation.csv
SCHEDULER_SERVICE_BASE_URL= /scheduler/
SCHEDULER_SERVICE_ERROR_REPORTING_EMAIL_ID= rakesh.k@pacewisdom.com
SCHEDULER_SERVICE_HOST= http://localhost:3567
SCHEDULER_SERVICE_URL= http://localhost:3567/jobs/scheduleJob
created_time= 2024-02-08T07:40:04.571464939Z
custom_metadata= null
destroyed= false
version= 31

Save and exit.

Setting up Databases

Log into the postgres user

sudo su postgres

Log into psql

psql -p 9700

Create a database user/role:

CREATE USER shikshalokam WITH ENCRYPTED PASSWORD 'slpassword';

Create the elevate_user database

CREATE DATABASE elevate_user;
GRANT ALL PRIVILEGES ON DATABASE elevate_user TO shikshalokam;
\c elevate_user
GRANT ALL ON SCHEMA public TO shikshalokam;

Running Migrations To Create Tables

Exit the postgres user account and install sequelize-cli globally

$ sudo npm i sequelize-cli -g

Navigate to the src folder of user service and run sequelize-cli migration command:

user/src$ npx sequelize-cli db:migrate

Now all the tables must be available in the Citus databases

Setting up Distribution Columns in Citus PostgreSQL Database

Refer Choosing Distribution Column for more information regarding Citus distribution columns.

Login into the postgres user

sudo su postgres

Login to psql

psql -p 9700

Login to the elevate_user database

\c elevate_user

Enable Citus for elevate_user

CREATE EXTENSION citus;

Within elevate_user, run the following queries:

SELECT create_distributed_table('entities', 'entity_type_id');
SELECT create_distributed_table('entity_types', 'organization_id');
SELECT create_distributed_table('file_uploads', 'organization_id');
SELECT create_distributed_table('forms', 'organization_id');
SELECT create_distributed_table('notification_templates', 'organization_id');
SELECT create_distributed_table('organizations', 'id');
SELECT create_distributed_table('organization_codes', 'code');
SELECT create_distributed_table('organization_domains', 'domain');
SELECT create_distributed_table('organization_role_requests','organization_id');
SELECT create_distributed_table('organization_user_invites','organization_id');
SELECT create_distributed_table('users_credentials','email');
SELECT create_distributed_table('users', 'organization_id');

Running Seeder to Populate the Tables with Seed Data

Exit the postgres user navigate to the script folder of the user service

Run the insertDefaultOrg.js script

src/scripts$ node insertDefaultOrg.js

Keep note of the default organization id generated by the script

Navigate to the src folder of the user service and update the .env file with these variables:

DEFAULT_ORG_ID=<id generated by the insertDefaultOrg script>
DEFAULT_ORGANISATION_CODE=default_code

Run the seeder command

src$ npm run db:seed:all

Start the Service

Run pm2 start command:

user/src$ pm2 start app.js -i 2 --name elevate-user

Run pm2 ls command

$ pm2 ls

Output should look like this (Sample output, might slightly differ in your installation):

┌────┬─────────────────────────┬─────────────┬─────────┬─────────┬──────────┬────────┬──────┬───────────┬──────────┬──────────┬──────────┬──────────┐
│ id │ name                    │ namespace   │ version │ mode    │ pid      │ uptime │ ↺    │ status    │ cpu      │ mem      │ user     │ watching │
├────┼─────────────────────────┼─────────────┼─────────┼─────────┼──────────┼────────┼──────┼───────────┼──────────┼──────────┼──────────┼──────────┤
│ 1  │ elevate-user            │ default     │ 1.0.0   │ cluster │ 106976   │ 27h    │ 0    │ online    │ 0%       │ 167.0mb  │ jenkins  │ disabled │
│ 2  │ elevate-user            │ default     │ 1.0.0   │ cluster │ 106986   │ 27h    │ 0    │ online    │ 0%       │ 169.3mb  │ jenkins  │ disabled │
└────┴─────────────────────────┴─────────────┴─────────┴─────────┴──────────┴────────┴──────┴───────────┴──────────┴──────────┴──────────┴──────────┘

This concludes the services and dependency setup.

Postman Collections

User Service

Migrations Commands

Check migrations

npm run elevate-migrations s

Create migrations

npm run elevate-migrations create categoryEntity #Where categoryEntity is the file name.

20220726145008-categoryEntity.js

We have followed the following structure for migration files to reduce code duplication.

let categories = [
	{
		value: 'sqaa',
		label: 'SQAA',
		image: 'entity/SQAA.jpg',
	},
	{
		value: 'communication',
		label: 'Communication',
		image: 'entity/Communication.png',
	},
    ...
]
var moment = require('moment')

module.exports = {
	async up(db) {
		global.migrationMsg = 'Uploaded categories entity'
		let entityData = []
		categories.forEach(async function (category) {
			category['status'] = 'ACTIVE'
			category['deleted'] = false
			category['type'] = 'categories'
			category['updatedAt'] = moment().format()
			category['createdAt'] = moment().format()
			category['createdBy'] = 'SYSTEM'
			category['updatedBy'] = 'SYSTEM'
			entityData.push(category)
		})
		await db.collection('entities').insertMany(entityData)
	},

	async down(db) {
		db.collection('entities').deleteMany({
			value: { $in: categories.map((category) => category.value) },
		})
	},
}

Run migrations

npm run elevate-migrations up

Down migrations

npm run elevate-migrations down

To know more about migrations refer project Wiki

Run tests

Integration tests

npm run test:integration

To know more about integration tests and their implementation refer to the project Wiki.

Unit tests

npm test

Used in

This project was built to be used with Mentor Service.

Notification service repo can be found here.

The PWA repo.

You can learn more about the full implementation of Mentor here .

Team

Open Source Dependencies

Several open source dependencies that have aided user service development:

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

README.md

README.md

User Service

System Requirements

Setup Options

Local Dependencies Steps

Remote Dependencies Steps

Installations

Install Node.js LTS

Install Build Essential

Install Kafka

Install Redis

Install Single-Node Citus (Distributed Database)

Install PM2

Setting up Repository

Clone the user repository to /opt/backend directory

Install Npm packages from src directory

Create .env file in src directory

Setting up Databases

Running Migrations To Create Tables

Setting up Distribution Columns in Citus PostgreSQL Database

Running Seeder to Populate the Tables with Seed Data

Start the Service

Run pm2 ls command

Postman Collections

Migrations Commands

Check migrations

Create migrations

Run migrations

Down migrations

Run tests

Integration tests

Unit tests

Used in

Team

Open Source Dependencies

Files

README.md

Latest commit

History

README.md

File metadata and controls

User Service

System Requirements

Setup Options

Local Dependencies Steps

Remote Dependencies Steps

Installations

Install Node.js LTS

Install Build Essential

Install Kafka

Install Redis

Install Single-Node Citus (Distributed Database)

Install PM2

Setting up Repository

Clone the user repository to /opt/backend directory

Install Npm packages from src directory

Create .env file in src directory

Setting up Databases

Running Migrations To Create Tables

Setting up Distribution Columns in Citus PostgreSQL Database

Running Seeder to Populate the Tables with Seed Data

Start the Service

Run pm2 ls command

Postman Collections

Migrations Commands

Check migrations

Create migrations

Run migrations

Down migrations

Run tests

Integration tests

Unit tests

Used in

Team

Open Source Dependencies