Skip to content

Commit

Permalink
Sync/2024 02 26 (#310)
Browse files Browse the repository at this point in the history
* fix dataset collaborator and issue delete authorization

* fix(map view): prevent client error when legend config is empty

* Feature/pending datasets (#198)

* Add pending_datasets table init and CRUD

* Add approval_status and draft to metadata

* Fix draft permissions

* [main.yml] Init. pendingdb tables

* Run WRI unit tests first

* Fix paths in unit tests script

* Run WRI unit tests first in src

* Fix OR in unit tests script

---------

Co-authored-by: Muhammad Ismail Shahzad <57398621+MuhammadIsmailShahzad@users.noreply.github.com>

* Rm test.yml (#202)

* [xl]: approval ui implementation

* add approval permission

* Normalize filenames instead of using a uuid (#205)

* approval workflow implementation

* add test

* fix test

* update permission

* add issue tab indexing

* update approval workflow

* update documentation

* display filled metadata fields

* Prefect datapusher (#208)

* Prefect datapusher

* Progress

* Datapusher

* Use default id

* Progress

* Datapusher ci/cd (#207)

* [main.yml] Add CI/CD for datapusher

* [values.yaml.dev.template] Add github sha for datapusher

* Progress

* Progress

* Progress

* Progress

* Fix build

* FInisihed

* Remove rm command

* Fix tests

* Fix tests

* Wait on prefect

* Fix typo

* Trigger CI

* Change viewport

* Update tests

* Update github action

* Fix tests

* Fix tests

* Fix tests

* Fix tests

* Fix tests

* Fix tests

* Fix tests

* Fix tests

* Fix tests

* Check logs

* Fix tests

* Fix tests

* Datapusher token

* Fix tests

* Fix tests

* Fix tests

* Init exteions

* Fix tests

* Fix tests

* Change permissions of file

* Fix tests

* Fix test

* Trigger CI

* Remove api_token

* Fix tests

* Fix build

* Update datapusher

* Fix tests

* Fix unit tests

* Fix test.ini

---------

Co-authored-by: Muhammad Ismail Shahzad <57398621+MuhammadIsmailShahzad@users.noreply.github.com>

* Return RW Error message (#210)

* Return RW Error message

* Fix build

* Trigger CI

* Fix file upload

* Fix datapusher and resource upload (#211)

* Improvements table view (#212)

* Improvements table view

* Datepicker on tabular view

* Default values (#213)

* Default values

* Fix pagination

* Enable other data formats (#215)

* Fix datafiles editing (#216)

* Feat/chart view connector url (#219)

* feat: basic chart creation functionallity

* feat: chart view creation/visualization

* feat(chart view): fix self is not defined error, imrpove formats select, responsiveness fixes

* fix(chart view): dybamic import chart

* fix(chart view): dybamic import chart

* feat(chat view): public view options

* feat(chat view): implement pie charts

* feat(chat view): implement embedding

* fix(chart view): minor fixes to the editor

* fix(chart view): show all format options

* feat(chart view): refactor components structure

* feat(chart view): makes it possible to create chart views based on connector url

* fix(chart view): linting errors

* fix(chart view): chart creation button (#222)

* QA Fixes (#223)

* Show datapusher status (#225)

* Show datapusher status

* Status on tab for datapusher

* Add spinner to scheduled

* Fix/chart view ii (#226)

* feat(chart view): allow users to see dataset charts

* feat(chart view): dataset views are now embedable

* feat(chart view): update resource and view update button label

* feat(chart view): show a tooltip with a message when chart view cannot be saved due to pending preview update

* fix(chart view): enable tooltips by default

* feat(chart view): start implementing chart view icons

* fix(chart view): fix page braking when maps are present

* fix build

* trigger build

* temporarily remove test

* Fix xlsx datapusher (#227)

* Fix datastore query (#228)

* Fix/chart view ii (#229)

* feat(chart view): allow users to see dataset charts

* feat(chart view): dataset views are now embedable

* feat(chart view): update resource and view update button label

* feat(chart view): show a tooltip with a message when chart view cannot be saved due to pending preview update

* fix(chart view): enable tooltips by default

* feat(chart view): start implementing chart view icons

* fix(chart view): fix page braking when maps are present

* fix build

* trigger build

* temporarily remove test

* build: add map test again

* build: remove temporary test

* feat(chart view): maintain flag on dataset that indicates whether it has chart views or not

* fix(chart view): do not allow category, measure and dimension to be the same

* fix(map view): fix tests

* build: add map test again

* fix(map view): fix tests

* fix(tests): remove unnecessary file

* fix build

* feat(chart view): add tests

* feat(chart view): add docs

* fix: remove unnecessary header from rw api

* fix(chart view): proxy requests to rw api (#230)

* Tabular preview fixes (#231)

* Tabular preview fixes

* Add back ()

* Trigger CI

* Fix/chart view iii (#234)

* fix(chart view): add loading indicator for view create and update + fix success notification theme

* fix(chart view): fix plot height issue when plot is refreshed

* fix(chart view): tweak sorting behavior

* Docs datapusher tabular preview (#232)

* Boot up staging env (#21)

* Add staging files
* Enable s3filestore and disable init cont.
* Initial frontend (#23)

---------

Co-authored-by: Luccas Mateus <luccasmmg@gmail.com>

* [hotfix] Fix cert manager name for ingress

* [main.yml] Fix frontend build secrets

* [cost-splitting] Change the links according to the branch

* [cost-splitting] Update branch name

* Avoid overwriting cost splitting files of staging

* Fix url for cypress tests

* Sync dev to staging and enable frontend deployment (#47)

* Updating the storage report

* Cost splitting will now run on different buckets (#24)

Depending on branch

* Updating the storage report

* Switch to cost splitting branch for s3filestore

* Updating the storage report

* Updating the storage report

* [Dockerfile] Enable s3filestore

* [values.yaml.dev.template] Enable s3filestore

* Normalize date string

* Updating the storage report

* Add all files

* Updating the storage report

* Fix typo

* Updating the storage report

* Get org title

* Updating the storage report

* Fix typo

* Updating the storage report

* Debug

* Updating the storage report

* Clear stuff

* Updating the storage report

* Updating the storage report

* Initial frontend (#25)

Co-authored-by: Luccas Mateus <luccasmmg@gmail.com>

* Rename the base directory as it includes frontend too now

* Use datapusher from the ecr

* Enable frontend and use images from ECR

* Enable frontend build and push

* [values.yaml.dev.temp] Add frontend ingress

* [main.yml] Fix CKAN_IMAGE for building

* [main.yml] Fix Trivy repo variable

* Add Site Title for frontend env

* Enable auth and fix frontend svc

* Run builds on dev and PRs to dev only

* Changing text color

* Fix cost splitting CI

* [images][l]: add images

* [package][s]: add headlessui

* [taildwind.config][m]: add custom themes

* [components][l]: implement shared and locallize components

* [footer][m]: implement footer component

* [Header][s]: implement header component

* [_documents][s]: add font url to document level

* [Search][m]: implement search page

* [search-page][s]: remove div and use fragment

* [fix][l]: update styling

* [package][s]: add swiper

* [Carousel][m]: add swiper carousel

* [carouselnavbutton][m]: add carousel nav button

* [Carousel][l]: wrap higlight and recetly sections inide carousel

* [Footer][s]: fix footer alignment issue

* [Header][s]: remove comment

* [Carousel][s]: remove state

* fix spacing to correct vieport

* Shell for search page

* Advanced search

* Fix build

* feat(odp-136): implement explore topics page UI

* fix(odp-136): eslint errors

* Little fixes

* Text-xs on small mobile

* Individual page for topic

* Copy button

* Acumin Pro used

* Add WRI extension and update CKAN package schema (#20)

* Sync dev with master (#19)

* Fix/broken build (#13)

* [cost-splitting-report.yml] Add permissions

* [values.yaml.dev.template] Add quotes to the bool env variables

---------

Co-authored-by: Muhammad Ismail Shahzad <57398621+MuhammadIsmailShahzad@users.noreply.github.com>

* Update cost-splitting-report.yml

---------

Co-authored-by: Luccas Mateus <Luccasmmg@gmail.com>

* Add wri extension and update package schema

* Add gitkeep for src directory

* Update gitignore

* Use symlink for ckanext-wri

* Fix tests

* [GitHub Actions] try ckan-dev instead of localhost

* [GitHub Actions] Add wait-on to docker step

* [GitHub Actions] Revert wait-on

* [GitHub Actions] Fix repo path

* [GitHub Actions] Install wri extension

* [GitHub Actions] Install wri extension

* [GitHub Actions] Install wri extension - fix path

* [GitHub Actions] Install wri extension - fix path and add APP_DIR

* [GitHub Actions] Install wri extension - try symlink in cwd

* [GitHub Actions] Install wri extension - remove symlink and update path

* [GitHub Actions] Install wri extension - add debug log

* [GitHub Actions] Install wri extension

* [GitHub Actions] Install wri extension

* [GitHub Actions] Install wri extension

* [GitHub Actions] Install wri extension

* [GitHub Actions] Install wri extension

* [GitHub Actions] Install wri extension

* [GitHub Actions] Install wri extension

* [GitHub Actions] Install wri extension - mv extension during action

* [GitHub Actions] Install wri extension - mv extension during action

* [GitHub Actions] Install wri extension - mv extension during action

* [GitHub Actions] Fix tests

* [GitHub Actions] Generate JWT tokens

* [GitHub Actions] Install openssl

* [GitHub Actions] Install openssl as root

* [GitHub Actions] Fix key generation

* [GitHub Actions] Fix key generation - remove pre-generated keys

* [GitHub Actions] Test adding plugins directly in test

* [GitHub Actions] Update scheming env variables

* Fix schema validators

* Fix schema validators

* Remove debugging output

* Dummy commit to run CI/CD

* Update docs

---------

Co-authored-by: Muhammad Ismail Shahzad <57398621+MuhammadIsmailShahzad@users.noreply.github.com>
Co-authored-by: Luccas Mateus <Luccasmmg@gmail.com>

* fix: minor typo on classname

* [Odp 147]:Team metadata page implementation [mockup implementation] (#35)

* [tailwindconfig][s]" add maxwidth

* add icon and image

* Team page implementation

* [TeamHeader][l]: update styling

* group component

* add activity icons

* [ActivityStream][l]: add activity stream components

* [DatasetList][l]: component to list all dataset

* [Teams][xl]: add teams page

* update teams page- move to a subpage

* update vertical space

* Little fixes

---------

Co-authored-by: Luccas Mateus <luccasmmg@gmail.com>

* build: trigger reubild

* feat(odp-137): implement UI for explore teams page

* Enable scheming extension on test site

* build: dummy commit to trigger build

* Dummy commit to rerun CI/CD

* Another dummy commit to rerun CI/CD

* Another dummy commit to rerun CI/CD

* Trigger CI

* build: dummy commit to trigger build

* Temporarily remove unit tests

* Fix schema tests

* Fixes odp-134 - Misc (#39)

* fixe alignment issues

* [Recent][s]:add shadows to bottom

* update styling

* Fix responsiveness

* Fix icons

* [TeamSearchResults][[s]: add pagination

* fix: active link effect and login button spacing on mobile (#41)

* fix: active link effect and login button spacing on mobile

* Change route name to match url

---------

Co-authored-by: Luccas Mateus <luccasmmg@gmail.com>

* Fix share topic button (#43)

* [values.yaml.dev.template] Increase frontend resources (#44)

* fix styling and spacing (#45)

* [main.yml] Fix frontend build secrets

* [cost-splitting] Change the links according to the branch

* [cost-splitting] Update branch name

* Avoid overwriting cost splitting files of staging

* Fix url for cypress tests

---------

Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Luccas Mateus <Luccasmmg@gmail.com>
Co-authored-by: Michael Polidori <contact@michaelpolidori.com>
Co-authored-by: steveoni <steohenoni2@gmail.com>
Co-authored-by: João Demenech <joao.demenech@datopian.com>
Co-authored-by: Demenech <joaommdemenech@gmail.com>

* [main.yml] Use same url for int tests as dev

* [codeql] Use staging branch

* Sync staging to dev (#105)

* Fix Staging Title

* Fix download link (#136) (#137)

* Fix download link

* Change update to patch

* [Bypassed][Dockerfile] Disable public signup

* Remove test.yml as not currently needed

* Merge Dev to Staging (#201)

* fix(map view): prevent client error when legend config is empty

* Feature/pending datasets (#198)

* Add pending_datasets table init and CRUD

* Add approval_status and draft to metadata

* Fix draft permissions

* [main.yml] Init. pendingdb tables

* Run WRI unit tests first

* Fix paths in unit tests script

* Run WRI unit tests first in src

* Fix OR in unit tests script

---------

Co-authored-by: Muhammad Ismail Shahzad <57398621+MuhammadIsmailShahzad@users.noreply.github.com>

* Rm test.yml (#202)

---------

Co-authored-by: Demenech <joaommdemenech@gmail.com>
Co-authored-by: João Demenech <joao.demenech@datopian.com>
Co-authored-by: Michael Polidori <contact@michaelpolidori.com>
Co-authored-by: Muhammad Ismail Shahzad <57398621+MuhammadIsmailShahzad@users.noreply.github.com>

* Datapusher docs

* Update docs

* Fix typo

* Add retry

* Fix test

* Fix tests

* Fix datapusher

* Fix types

* Revert "Merge branch 'staging' of https://github.com/wri/wri-odp into docs-datapusher-tabular-preview"

This reverts commit 2d72215, reversing
changes made to b5b90fa.

---------

Co-authored-by: Muhammad Ismail Shahzad <57398621+MuhammadIsmailShahzad@users.noreply.github.com>
Co-authored-by: MuhammadIsmailShahzad <ismailshahzadmirza@gmail.com>
Co-authored-by: GitHub Action <action@github.com>
Co-authored-by: Michael Polidori <contact@michaelpolidori.com>
Co-authored-by: steveoni <steohenoni2@gmail.com>
Co-authored-by: João Demenech <joao.demenech@datopian.com>
Co-authored-by: Demenech <joaommdemenech@gmail.com>

* update activity stream to fetch filter data via server request (#241)

* Make datapusher not conflict with multiple environmetns (#243)

* Make datapusher not conflict with multiple environmetns

* fix typo

* Trigger CI (#244)

* Trigger CI

* Trigger CI

* Add retries

* Fix: Create resource view for non-public datasets not working (#245)

* fix(chart view): auth header not beingproperly passed

* trigger build

* Feat/api endpoints (#246)

* feat: add API endpoints information to dataset and datafile metadata sections

* feat(docs): add API endpoints docs

* feat(tests): add a test case for API endpoints

* fix: minor issue with syntax highlighting

* fix: missing dependency

* fix(API endpoints): prevent double slashes on endpoints URLs (#248)

* Decoded layers start (#249)

* approval workflow createform (#224)

* approval workflow createform

* update approval create form

* update on edit andtabular resources

* edit datasete

* update approval workflow: create and edit form

* fix test

* fix test

* fix type and data visibility

* fix test

* update documentation

* update test

* update test

* update test

* fix test

* update test

* update test

* fix test

* increase timeout

* fix test

* Odp 197 (#250)

* Progress

* Progress

* ODP-197

* Fix datapusher test

* Fix build

* Fix tests

* Fix tests

* fix wrong table fields (#251)

* Manage how data enters approval state (#253)

* fix wrong table fields

* filter out updated value in editform

* filter updated fields

* Documentation ODP-197 (#252)

* Documentation ODP-197

* Add animation

* Feat/file conversion (#254)

* feat(dataset page): refactor datafile download button

* feat(dataset page): add datafile download dropdown

* feat: converted data files download

* feat(downloads): add docs

* fix: update envvars for dev/tests envs

* fix lint

* Dummy commit

* fix(file download): fix expiration time for signed URL + missing key in download_callback (#255)

* disable notification for private and draft (#256)

* Decode functions second try (#257)

* QA Fixes (#258)

* QA Fixes

* Trigger CI

* Fix dataset creation (#259)

* Make tests more robust (#260)

* Fix dataset creation

* Make tests more robus

* Fix/file conversion qa (#261)

* fix(file download): make layers use the layer SQL

* feat(views): tweak add view button labels

* fix(file download): make filenames human-readable

* feat(views): create chart button now shows a tooltip when chart is not creatable i.e. data file is not pushed to DataStore

* fix(file download): sanitize headers when converting to XML

* fix(file download): fix ODP link in the download file email template

* fix(views): fix test that checks whether the View Chart Preview button is present

* fix(views): fix test that checks whether the View Table Preview button is present

* fix(file download): fix SQL query for Layers

* fix(file download): add simple test

* Change text (#262)

* fixes based on qa (#263)

* disable notification for private and draft

* fix  chart view test

* update based on qa

* fix type error

* fix test

* increase timeout

* fix reject approval

* fix test

* increase timeout

* comment out banner check

* QA fixes - Update views and data files (#265)

* fix(resource edit): fix syntax error

* fix(view edit): redirect after save not working when editing layers + remove console logs

* fix(dataset create test): commented out line that checks notification after editing dataset because of redirect

* fix: increate retries for chart view test

* manage approval dataset display and qa fixes (#266)

* manage approval public and fix qa

* update pensing data display

* fix test

* update workflow

* fix build

* Decoded functions 3 (#264)

* Change text

* Decoded functions 3

* Fix tooltip interaction form (#267)

* fix approval issues (#268)

* fix approval issues

* add sysadmin check

* fix topics not displaying

* fix test

* Fix client errro feb 12 (#269)

* GLAD Alerts (#273)

* feat(API endpoints): implement code snippets for Python and R (#272)

* fix(file download): add authorization header when requesting to download a file from a private dataset (#275)

* fix page content not loading on navigation [ODP-199] (#274)

* fix page content not loading on navigation

* moved key from global to local scope of teams and topics

* Approval workflow for private datasets (#277)

* Approval workflow for private datasets

* Trigger CI

* Trigger CI

* Fixes/fe improvements (#278)

* fix(maps): icon for hide/show layer not changing according to layer state

* fix(maps): legends toggle arrow alignment

* feat(sign in): redirect to dashbord after sign in

* fix(sign in): when signing in with azure, show loading indicator on the correct button

* trigger build

* Fix inconsistent number of dataset under topics and teams (#279)

* fix page content not loading on navigation

* get the right number of datset and count

* Edited alerts (background change) (#280)

* Fix licenses (#281)

* Fix datafiles highlight (#283)

* Fix null user on approvalrow component (#282)

* Prettier diff (#285)

* Prettier diff

* Trigger CI

* Replace resources/organization with datafiles/teams

* Fix tests

* Try to fix notifications (#287)

* Try to fix notifications

* Fix build

* Change title

* Fix dataset pushed to pending when collaborator is added (#288)

* fix page content not loading on navigation

* fix collaborator fields not saved alone

* [Odp 203]: Default data preview (#286)

* fix page content not loading on navigation

* add default view

* fix types

* update default view

* fix double rendring of smae layer

* fix dataset version

* remove preview tab

* Only show featured for public datasets (#289)

* Only show featured for public datasets

* only show featured when public is selected

* feat(release notes): implementation + docs + tests (#291)

* Fix changing resource title name (#290)

* Fix logan (#292)

* Fix logan

* Fix resources not being created

* Seo - [ODP-23] (#293)

* fix page content not loading on navigation

* add seo tags

* fix invlaid config

* add next-sitemap

* add seo config

* Fix datapreview logic (#295)

* fix page content not loading on navigation

* fix data preview logic

* Release Notes QA Fixes (#297)

* fix(release notes): make release notes field RTE + fix scrollbars

* fix(tests): update tests to work with new release notes editor

* fix(tests): update tests to work with new release notes editor

* fix(tests): update tests to work with new release notes editor

* display tags and topics (#296)

* display tags and topics

* Trigger CI

* Fix tests

* Fix tests

* Fix tests

* Fix unit tests

* Fix tabular preview taking too long to load, due to waiting for session apikey to get resolve on initial load (#299)

* fix page content not loading on navigation

* fix preview wating for session to get resolve

* fix(release notes): display release notes in descending order (#301)

* Fix dataset being just created (#302)

Co-authored-by: Luccas Mateus <luccasmmg@gmail.com>

* Disable datapusher for links (#300)

* Disable datapusher for links

* Fix typo

* Fix typo

* Show datasettitle on toaster

* Use url_type instead of type

* Fix types

* Trigger CI

* Fix tests

* Fix tests

* Fix tests

* Fix tests

* Fix tests

* Fix tests

* Fix tests

* Fix conversion test

* More domains

* Fix usernames in notification emails

* Fix bread crumb (#306)

* fix(release notes): fix preview when creating dataset + fix order (#308)

* fix(release notes): fix preview when creating dataset + fix order

* fix(release notes): fix release notes rendering wrong on approval row

* Fix/release notes general (#309)

* fix(release notes): fix preview when creating dataset + fix order

* fix(release notes): fix release notes rendering wrong on approval row

* fix(release notes): fix release notes not showing for newly created datasets + minor rendering issue

---------

Co-authored-by: steveoni <steohenoni2@gmail.com>
Co-authored-by: Michael Polidori <contact@michaelpolidori.com>
Co-authored-by: Muhammad Ismail Shahzad <57398621+MuhammadIsmailShahzad@users.noreply.github.com>
Co-authored-by: Luccas Mateus <Luccasmmg@gmail.com>
Co-authored-by: MuhammadIsmailShahzad <ismailshahzadmirza@gmail.com>
Co-authored-by: GitHub Action <action@github.com>
  • Loading branch information
7 people authored Feb 26, 2024
1 parent c88fb5b commit 1b153b0
Show file tree
Hide file tree
Showing 53 changed files with 848 additions and 314 deletions.
3 changes: 2 additions & 1 deletion .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ jobs:
--build-arg NEXTAUTH_URL=${{ secrets.STAGING_FRONTEND_NEXTAUTH_URL }} \
--build-arg CKAN_URL=${{ secrets.STAGING_FRONTEND_CKAN_URL }} \
--build-arg NEXT_PUBLIC_CKAN_URL=${{ secrets.STAGING_FRONTEND_CKAN_URL }} \
--build-arg NEXT_PUBLIC_NEXTAUTH_URL=${{ secrets.STAGING_FRONTEND_NEXTAUTH_URL }} \
deployment/frontend
docker push $REGISTRY/$FRONTEND_REPO:$IMAGE_TAG
- name: Build and push Datapusher image to ECR
Expand Down Expand Up @@ -248,4 +249,4 @@ jobs:
- name: 'Deploy using Helm Upgrade'
run: |
set -e
helm upgrade -i dx-helm-wri-$BRANCH_NAME-release ./deployment/helm-templates -f ./deployment/helm-templates/values.yaml -n $PROJECT_NAME-$BRANCH_NAME --create-namespace --wait
helm upgrade -i dx-helm-wri-$BRANCH_NAME-release ./deployment/helm-templates -f ./deployment/helm-templates/values.yaml -n $PROJECT_NAME-$BRANCH_NAME --create-namespace --wait
1 change: 1 addition & 0 deletions ckan-backend-dev/.env.example
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,7 @@ NEXTAUTH_SECRET=secret
NEXTAUTH_URL=http://frontend:3000
CKAN_URL=http://ckan-dev:5000
NEXT_PUBLIC_CKAN_URL=http://ckan-dev:5000
NEXT_PUBLIC_NEXTAUTH_URL=http://frontend:3000
S3_ACCESS_KEY_ID=${MINIO_ROOT_USER}
S3_SECRET_KEY_ID=${MINIO_ROOT_PASSWORD}
S3_BUCKET_NAME=ckan
Expand Down
1 change: 1 addition & 0 deletions ckan-backend-dev/docker-compose.test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ services:
context: ../deployment/frontend/
args:
- NEXT_PUBLIC_CKAN_URL=http://ckan-dev:5000
- NEXT_PUBLIC_NEXTAUTH_URL=http://localhost:3000
environment:
- NEXTAUTH_SECRET=secret
- NEXTAUTH_URL=http://localhost:3000
Expand Down
18 changes: 13 additions & 5 deletions ckan-backend-dev/src/ckanext-wri/ckanext/wri/logic/action/get.py
Original file line number Diff line number Diff line change
Expand Up @@ -481,6 +481,7 @@ def pending_diff_show(context: Context, data_dict: DataDict):

dataset_diff = None

pending_dataset = None
try:
pending_dataset = PendingDatasets.get(package_id=package_id)
log.error("===============pending dataset=================")
Expand All @@ -499,7 +500,11 @@ def pending_diff_show(context: Context, data_dict: DataDict):
# _("Diff not found for Pending Dataset: {}".format(package_id))
# )

return dataset_diff
return {
"diff": dataset_diff,
"old_dataset": existing_dataset,
"new_dataset": pending_dataset,
}


def _diff(existing, pending, path=""):
Expand Down Expand Up @@ -542,13 +547,15 @@ def _process_lists(existing_list, pending_list, path):

return list_diff


@logic.side_effect_free
def dataset_release_notes(context: Context, data_dict: DataDict):
model = context["model"]
conn = model.Session.connection()

id = data_dict.get("id")
sql = text('''
sql = text(
"""
select
distinct on ((data::json->>'package')::json->>'release_notes')
((data::json->>'package')::json->>'release_notes') as release_notes,
Expand All @@ -561,11 +568,12 @@ def dataset_release_notes(context: Context, data_dict: DataDict):
and ((data::json->>'package')::json->>'release_notes') != ''
order by
1, 2 DESC
'''.format(id))
""".format(
id
)
)
q = conn.execute(sql)

results = q.all()

return results


1 change: 1 addition & 0 deletions ckan-backend-dev/src/ckanext-wri/ckanext/wri/plugin.py
Original file line number Diff line number Diff line change
Expand Up @@ -210,6 +210,7 @@ def _submit_to_datapusher(self, resource_dict: dict[str, Any]):
resource_format
and resource_format.lower() in supported_formats
and resource_dict.get('url_type') != u'datapusher'
and resource_dict.get('url_type') == u'upload'
)

if not submit:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -114,10 +114,11 @@ def test_pending_diff_show():

get_action("pending_dataset_update")(context, data_dict)

result = get_action("pending_diff_show")(context, {"package_id": dataset["id"]})
_result = get_action("pending_diff_show")(context, {"package_id": dataset["id"]})
result = _result['diff']

assert result["title"]["new_value"] == "New Title"
assert result["notes"]["new_value"] == "New description"
assert result["private"]["new_value"] is True
assert result["rw_dataset"]["new_value"] is False
assert result["wri_data"]["new_value"] is True
assert result["wri_data"]["new_value"] is True
1 change: 1 addition & 0 deletions deployment/frontend/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ ARG NEXTAUTH_SECRET
ARG NEXTAUTH_URL
ARG CKAN_URL
ARG NEXT_PUBLIC_CKAN_URL
ARG NEXT_PUBLIC_NEXTAUTH_URL
ARG AZURE_AD_CLIENT_ID
ARG AZURE_AD_CLIENT_SECRET
ARG AZURE_AD_TENANT_ID
Expand Down
22 changes: 22 additions & 0 deletions deployment/frontend/next-sitemap.config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
console.log('[*] SITE_URL', process.env.NEXT_PUBLIC_NEXTAUTH_URL)

module.exports = {
siteUrl: process.env.NEXT_PUBLIC_NEXTAUTH_URL,
generateRobotsTxt: true,
exclude: ['/server-sitemap.xml', '/dashboard/*', '/dashboard'],
robotsTxtOptions: {
additionalSitemaps: [
`${process.env.NEXT_PUBLIC_NEXTAUTH_URL}/server-sitemap.xml`,
],
policies: [
{
userAgent: '*',
disallow: ['/dashboard/*']
},
{
userAgent: '*',
allow: '/',
}
],
}
};
58 changes: 33 additions & 25 deletions deployment/frontend/next.config.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -3,37 +3,45 @@
* for Docker builds.
*/

import withBundleAnalyzer from '@next/bundle-analyzer';
import withBundleAnalyzer from '@next/bundle-analyzer'

await import("./src/env.mjs");
await import('./src/env.mjs')

const bundleAnalyzer = withBundleAnalyzer({
enabled: process.env.ANALYZE === 'true',
enabled: process.env.ANALYZE === 'true',
})

/** @type {import("next").NextConfig} */
const config = {
eslint: {
// Warning: This allows production builds to successfully complete even if
// your project has ESLint errors.
ignoreDuringBuilds: true,
},
reactStrictMode: true,
output: "standalone",
eslint: {
// Warning: This allows production builds to successfully complete even if
// your project has ESLint errors.
ignoreDuringBuilds: true,
},
reactStrictMode: true,
output: 'standalone',

/**
* If you are using `appDir` then you must comment the below `i18n` config out.
*
* @see https://github.com/vercel/next.js/issues/41980
*/
i18n: {
locales: ['en'],
defaultLocale: 'en',
},
images: {
domains: [
'wri-dev-ckan-svc',
'wri.dev.ckan.datopian.com',
'wri.staging.ckan.datopian.com',
'wri.prod.ckan.datopian.com',
'test-bucket-wri.s3.ap-northeast-1.amazonaws.com',
'ckan-dev',
'gravatar.com',
's3.amazonaws.com',
],
},
}

/**
* If you are using `appDir` then you must comment the below `i18n` config out.
*
* @see https://github.com/vercel/next.js/issues/41980
*/
i18n: {
locales: ["en"],
defaultLocale: "en",
},
images: {
domains: ['wri-dev-ckan-svc', 'wri.dev.ckan.datopian.com', 'test-bucket-wri.s3.ap-northeast-1.amazonaws.com', 'ckan-dev', 'gravatar.com', 's3.amazonaws.com'],
},
};

export default bundleAnalyzer(config);
export default bundleAnalyzer(config)
58 changes: 58 additions & 0 deletions deployment/frontend/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 5 additions & 1 deletion deployment/frontend/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@
"lint": "next lint",
"start": "next start",
"test": "vitest",
"test-ci": "vitest run"
"test-ci": "vitest run",
"postbuild": "next-sitemap"
},
"dependencies": {
"@aws-sdk/client-s3": "^3.441.0",
Expand Down Expand Up @@ -41,11 +42,13 @@
"@tanstack/react-query": "^4.32.6",
"@tanstack/react-table": "^8.10.7",
"@tiptap/extension-bold": "^2.1.12",
"@tiptap/extension-bullet-list": "^2.2.3",
"@tiptap/extension-code": "^2.1.12",
"@tiptap/extension-document": "^2.1.12",
"@tiptap/extension-history": "^2.1.12",
"@tiptap/extension-italic": "^2.1.12",
"@tiptap/extension-link": "^2.1.12",
"@tiptap/extension-list-item": "^2.2.3",
"@tiptap/extension-paragraph": "^2.1.12",
"@tiptap/extension-placeholder": "^2.1.13",
"@tiptap/extension-strike": "^2.1.12",
Expand Down Expand Up @@ -83,6 +86,7 @@
"next": "^13.5.4",
"next-auth": "^4.24.5",
"next-seo": "^6.1.0",
"next-sitemap": "^4.2.3",
"nodemailer": "^6.7.8",
"old-vizzuality-components": "npm:vizzuality-components@1.3.0",
"papaparse": "^5.4.1",
Expand Down
7 changes: 7 additions & 0 deletions deployment/frontend/public/robots.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Block all crawlers for /dashboard
User-agent: *
Disallow: /dashboard

# Allow all crawlers
User-agent: *
Allow: /
Loading

0 comments on commit 1b153b0

Please sign in to comment.