Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Main #263

Merged
merged 5 commits into from
Jun 21, 2024
Merged

Main #263

Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 2 additions & 30 deletions docs/connectors/connectors.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,42 +96,14 @@ copy token from response
}
}
```

#### Microsoft Teams

```json
{
"channel": "",
"topics": ["",""],
"files": {
"folder": "",
"recursive": false,
},
"token": {
"access_token": "",
"expiry": "",
"refresh_token": "",
"token_type": ""
}
}
```

- channel : name of channel for analyzing
- topics : false - scan only given folder , true - scan nested folders
- token : OAuth token for access to ```one drive```
- files :
- folder : optional, folder name for scanning
- recursive : false - scan only given folder , true - scan nested folders


#### scope for oauth service

- ChannelMessage.Read.All
- Chat.Read Chat.ReadBasic
- Chat.Read
- Chat.ReadBasic
- Team.ReadBasic.All
- TeamSettings.Read.All
- ChannelSettings.Read.All
- Channel.ReadBasic.All
- Group.Read.All
- Directory.Read.All

8 changes: 3 additions & 5 deletions docs/connectors/teams.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,7 @@ At step 2:

```json
{
"channel": "",
"topics": [],
"team": "",
"files": {
"folder": "",
"recursive": true
Expand All @@ -28,9 +27,8 @@ At step 2:
}
```

- channel : name of channel for analyzing
- recursive : false - scan only given folder , true - scan nested folders
- token : OAuth token for access to ```one drive```
- team : name of team for analyzing
- token : OAuth token for access to ```ms teams```
- files :
- folder : optional, folder name for scanning
- recursive : false - scan only given folder , true - scan nested folders
Expand Down
14 changes: 13 additions & 1 deletion docs/microsoft/application-registration.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,4 +41,16 @@ then choose Save.
- offline_access
- Authentication
- Add platform
- choose web and define redirect URL https://rag.cognix.ch/api/oauth/microsoft/callback
- choose web and define redirect URL https://rag.cognix.ch/api/oauth/microsoft/callback
#### Scope for MS TEAMS

- ChannelMessage.Read.All
- Chat.Read
- Chat.ReadBasic
- Team.ReadBasic.All
- TeamSettings.Read.All
- ChannelSettings.Read.All
- Channel.ReadBasic.All
- Group.Read.All
- Directory.Read.All

7 changes: 7 additions & 0 deletions src/backend/connector/executor.go
Original file line number Diff line number Diff line change
Expand Up @@ -78,12 +78,15 @@ func (e *Executor) runConnector(ctx context.Context, msg jetstream.Msg) error {
// execute connector
resultCh := connectorWF.Execute(ctx, trigger.Params)
// read result from channel
hasSemanticMessage := false
for result := range resultCh {
var loopErr error
// empty result when channel was closed.
if result.SourceID == "" {
break
}
hasSemanticMessage = true

// save content in minio
if result.Content != nil {
if err = e.saveContent(ctx, result); err != nil {
Expand Down Expand Up @@ -139,6 +142,10 @@ func (e *Executor) runConnector(ctx context.Context, msg jetstream.Msg) error {
if err != nil {
zap.S().Errorf("failed to update documents: %v", err)
connectorModel.Status = model.ConnectorStatusUnableProcess
} else {
if !hasSemanticMessage {
connectorModel.Status = model.ConnectorStatusSuccess
}
}
connectorModel.LastUpdate = pg.NullTime{time.Now().UTC()}

Expand Down
5 changes: 2 additions & 3 deletions src/backend/core/connector/microsoft-core/drive.go
Original file line number Diff line number Diff line change
Expand Up @@ -94,11 +94,10 @@ func (c *MSDrive) getFile(item *DriveChildBody) error {
} else {
// when file was stored in minio URL should be minio:bucket:filename
minioFile := strings.Split(doc.URL, ":")
if len(minioFile) != 3 {
return fmt.Errorf("invalid file url: %s", doc.URL)
if len(minioFile) == 3 && minioFile[0] == "minio" {
fileName = minioFile[2]
}
// use previous file name for update file in minio
fileName = minioFile[2]
}
doc.IsExists = true

Expand Down
Loading
Loading