Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add account details to queryStatuses #3236

Open
wants to merge 25 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 23 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
7b7aad0
Moderation subjectStatusView with statistics
matthieusieben Dec 18, 2024
3f2e708
add account stats to queue entries
matthieusieben Dec 20, 2024
da81bb8
fix
matthieusieben Jan 10, 2025
70052ef
adapt
matthieusieben Jan 10, 2025
db29b49
Merge remote-tracking branch 'origin/main' into msi/ozone-subject-stats
matthieusieben Jan 14, 2025
76b1f85
review changes
matthieusieben Jan 14, 2025
8ac6d9b
style
matthieusieben Jan 14, 2025
30fd511
review comments
matthieusieben Jan 14, 2025
bffd74c
Merge remote-tracking branch 'origin/main' into msi/ozone-subject-stats
matthieusieben Jan 16, 2025
2cd6036
wip
matthieusieben Jan 16, 2025
b6fcd1e
add indexes
matthieusieben Jan 16, 2025
444f072
refresh materialized view
matthieusieben Jan 16, 2025
55d90a0
use the background queue to refresh materialized views
matthieusieben Jan 17, 2025
7759a30
tidy
matthieusieben Jan 17, 2025
b38d597
fix verrify
matthieusieben Jan 17, 2025
6685bea
Merge remote-tracking branch 'origin/main' into msi/ozone-subject-stats
matthieusieben Jan 17, 2025
1d8757d
tidy
matthieusieben Jan 17, 2025
5d3d829
Add filtering based on "minAccountSuspendCount"
matthieusieben Jan 17, 2025
3b09a4a
fix tests
matthieusieben Jan 17, 2025
77e3924
Update test snapshots
matthieusieben Jan 17, 2025
41924b7
update materialized views when running `processAll`
matthieusieben Jan 17, 2025
5983e5f
tidy
matthieusieben Jan 17, 2025
5fac972
processAll sequentially
matthieusieben Jan 17, 2025
38d22c1
tidy
matthieusieben Jan 17, 2025
71f8af2
tidy
matthieusieben Jan 17, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 74 additions & 0 deletions lexicons/tools/ozone/moderation/defs.json
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,80 @@
"tags": {
"type": "array",
"items": { "type": "string" }
},
"accountStats": {
"description": "Statistics related to the account subject",
"type": "ref",
"ref": "#accountStats"
},
"recordsStats": {
"description": "Statistics related to the record subjects authored by the subject's account",
"type": "ref",
"ref": "#recordsStats"
}
}
},
"accountStats": {
"description": "Statistics about a particular account subject",
"type": "object",
"properties": {
"reportCount": {
"description": "Total number of reports on the account",
"type": "integer"
},
"appealCount": {
"description": "Total number of appeals against a moderation action on the account",
"type": "integer"
},
"suspendCount": {
"description": "Number of times the account was suspended",
"type": "integer"
},
"escalateCount": {
"description": "Number of times the account was escalated",
"type": "integer"
},
"takedownCount": {
"description": "Number of times the account was taken down",
"type": "integer"
}
}
},
"recordsStats": {
"description": "Statistics about a set of record subject items",
"type": "object",
"properties": {
"totalReports": {
"description": "Cumulative sum of the number of reports on the items in the set",
"type": "integer"
},
"reportedCount": {
"description": "Number of items that were reported at least once",
"type": "integer"
},
"escalatedCount": {
"description": "Number of items that were escalated at least once",
"type": "integer"
},
"appealedCount": {
"description": "Number of items that were appealed at least once",
"type": "integer"
},
"subjectCount": {
"description": "Total number of item in the set",
"type": "integer"
},
"pendingCount": {
"description": "Number of item currently in \"reviewOpen\" or \"reviewEscalated\" state",
"type": "integer"
},
"processedCount": {
"description": "Number of item currently in \"reviewNone\" or \"reviewClosed\" state",
"type": "integer"
},
"takendownCount": {
"description": "Number of item currently taken down",
"type": "integer"
}
}
},
Expand Down
19 changes: 18 additions & 1 deletion lexicons/tools/ozone/moderation/queryStatuses.json
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,12 @@
"sortField": {
"type": "string",
"default": "lastReportedAt",
"enum": ["lastReviewedAt", "lastReportedAt"]
"enum": [
"lastReviewedAt",
"lastReportedAt",
"reportedRecordsCount",
"takendownRecordsCount"
]
},
"sortDirection": {
"type": "string",
Expand Down Expand Up @@ -158,6 +163,18 @@
"type": "string",
"description": "If specified, subjects of the given type (account or record) will be returned. When this is set to 'account' the 'collections' parameter will be ignored. When includeAllUserRecords or subject is set, this will be ignored.",
"knownValues": ["account", "record"]
},
"minAccountSuspendCount": {
"type": "integer",
"description": "If specified, only subjects that belong to an account that has at least this many suspensions will be returned."
},
"minReportedRecordsCount": {
"type": "integer",
"description": "If specified, only subjects that belong to an account that has at least this many reported records will be returned."
},
"minTakendownRecordsCount": {
"type": "integer",
"description": "If specified, only subjects that belong to an account that has at least this many taken down records will be returned."
}
}
},
Expand Down
101 changes: 100 additions & 1 deletion packages/api/src/client/lexicons.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11299,6 +11299,85 @@ export const schemaDict = {
type: 'string',
},
},
accountStats: {
description: 'Statistics related to the account subject',
type: 'ref',
ref: 'lex:tools.ozone.moderation.defs#accountStats',
},
recordsStats: {
description:
"Statistics related to the record subjects authored by the subject's account",
type: 'ref',
ref: 'lex:tools.ozone.moderation.defs#recordsStats',
},
},
},
accountStats: {
description: 'Statistics about a particular account subject',
type: 'object',
properties: {
reportCount: {
description: 'Total number of reports on the account',
type: 'integer',
},
appealCount: {
description:
'Total number of appeals against a moderation action on the account',
type: 'integer',
},
suspendCount: {
description: 'Number of times the account was suspended',
type: 'integer',
},
escalateCount: {
description: 'Number of times the account was escalated',
type: 'integer',
},
takedownCount: {
description: 'Number of times the account was taken down',
type: 'integer',
},
},
},
recordsStats: {
description: 'Statistics about a set of record subject items',
type: 'object',
properties: {
totalReports: {
description:
'Cumulative sum of the number of reports on the items in the set',
type: 'integer',
},
reportedCount: {
description: 'Number of items that were reported at least once',
type: 'integer',
},
escalatedCount: {
description: 'Number of items that were escalated at least once',
type: 'integer',
},
appealedCount: {
description: 'Number of items that were appealed at least once',
type: 'integer',
},
subjectCount: {
description: 'Total number of item in the set',
type: 'integer',
},
pendingCount: {
description:
'Number of item currently in "reviewOpen" or "reviewEscalated" state',
type: 'integer',
},
processedCount: {
description:
'Number of item currently in "reviewNone" or "reviewClosed" state',
type: 'integer',
},
takendownCount: {
description: 'Number of item currently taken down',
type: 'integer',
},
},
},
subjectReviewState: {
Expand Down Expand Up @@ -12547,7 +12626,12 @@ export const schemaDict = {
sortField: {
type: 'string',
default: 'lastReportedAt',
enum: ['lastReviewedAt', 'lastReportedAt'],
enum: [
'lastReviewedAt',
'lastReportedAt',
'reportedRecordsCount',
'takendownRecordsCount',
],
},
sortDirection: {
type: 'string',
Expand Down Expand Up @@ -12602,6 +12686,21 @@ export const schemaDict = {
"If specified, subjects of the given type (account or record) will be returned. When this is set to 'account' the 'collections' parameter will be ignored. When includeAllUserRecords or subject is set, this will be ignored.",
knownValues: ['account', 'record'],
},
minAccountSuspendCount: {
type: 'integer',
description:
'If specified, only subjects that belong to an account that has at least this many suspensions will be returned.',
},
minReportedRecordsCount: {
type: 'integer',
description:
'If specified, only subjects that belong to an account that has at least this many reported records will be returned.',
},
minTakendownRecordsCount: {
type: 'integer',
description:
'If specified, only subjects that belong to an account that has at least this many taken down records will be returned.',
},
},
},
output: {
Expand Down
62 changes: 62 additions & 0 deletions packages/api/src/client/types/tools/ozone/moderation/defs.ts
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,8 @@ export interface SubjectStatusView {
appealed?: boolean
suspendUntil?: string
tags?: string[]
accountStats?: AccountStats
recordsStats?: RecordsStats
[k: string]: unknown
}

Expand All @@ -151,6 +153,66 @@ export function validateSubjectStatusView(v: unknown): ValidationResult {
return lexicons.validate('tools.ozone.moderation.defs#subjectStatusView', v)
}

/** Statistics about a particular account subject */
export interface AccountStats {
/** Total number of reports on the account */
reportCount?: number
/** Total number of appeals against a moderation action on the account */
appealCount?: number
/** Number of times the account was suspended */
suspendCount?: number
/** Number of times the account was escalated */
escalateCount?: number
/** Number of times the account was taken down */
takedownCount?: number
[k: string]: unknown
}

export function isAccountStats(v: unknown): v is AccountStats {
return (
isObj(v) &&
hasProp(v, '$type') &&
v.$type === 'tools.ozone.moderation.defs#accountStats'
)
}

export function validateAccountStats(v: unknown): ValidationResult {
return lexicons.validate('tools.ozone.moderation.defs#accountStats', v)
}

/** Statistics about a set of record subject items */
export interface RecordsStats {
/** Cumulative sum of the number of reports on the items in the set */
totalReports?: number
/** Number of items that were reported at least once */
reportedCount?: number
/** Number of items that were escalated at least once */
escalatedCount?: number
/** Number of items that were appealed at least once */
appealedCount?: number
/** Total number of item in the set */
subjectCount?: number
/** Number of item currently in "reviewOpen" or "reviewEscalated" state */
pendingCount?: number
/** Number of item currently in "reviewNone" or "reviewClosed" state */
processedCount?: number
/** Number of item currently taken down */
takendownCount?: number
[k: string]: unknown
}

export function isRecordsStats(v: unknown): v is RecordsStats {
return (
isObj(v) &&
hasProp(v, '$type') &&
v.$type === 'tools.ozone.moderation.defs#recordsStats'
)
}

export function validateRecordsStats(v: unknown): ValidationResult {
return lexicons.validate('tools.ozone.moderation.defs#recordsStats', v)
}

export type SubjectReviewState =
| 'lex:tools.ozone.moderation.defs#reviewOpen'
| 'lex:tools.ozone.moderation.defs#reviewEscalated'
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,11 @@ export interface QueryParams {
ignoreSubjects?: string[]
/** Get all subject statuses that were reviewed by a specific moderator */
lastReviewedBy?: string
sortField?: 'lastReviewedAt' | 'lastReportedAt'
sortField?:
| 'lastReviewedAt'
| 'lastReportedAt'
| 'reportedRecordsCount'
| 'takendownRecordsCount'
sortDirection?: 'asc' | 'desc'
/** Get subjects that were taken down */
takendown?: boolean
Expand All @@ -62,6 +66,12 @@ export interface QueryParams {
collections?: string[]
/** If specified, subjects of the given type (account or record) will be returned. When this is set to 'account' the 'collections' parameter will be ignored. When includeAllUserRecords or subject is set, this will be ignored. */
subjectType?: 'account' | 'record' | (string & {})
/** If specified, only subjects that belong to an account that has at least this many suspensions will be returned. */
minAccountSuspendCount?: number
/** If specified, only subjects that belong to an account that has at least this many reported records will be returned. */
minReportedRecordsCount?: number
/** If specified, only subjects that belong to an account that has at least this many taken down records will be returned. */
minTakendownRecordsCount?: number
}

export type InputSchema = undefined
Expand Down
1 change: 1 addition & 0 deletions packages/dev-env/src/bin.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ const run = async () => {
port: 2587,
chatUrl: 'http://localhost:2590', // must run separate chat service
chatDid: 'did:example:chat',
dbMaterializedViewRefreshIntervalMs: 30_000,
},
introspect: { port: 2581 },
})
Expand Down
Loading