Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: Support multiple backends (4store, GraphDb, Virtuoso, and AllegroGraph) #152

Draft
wants to merge 251 commits into
base: develop
Choose a base branch
from

Conversation

syphax-bouazzouni
Copy link

@syphax-bouazzouni syphax-bouazzouni commented Apr 17, 2024

Require

Context

This PR makes the ths following triple stores: 4store (fs), GraphDb (gb), Virtuoso (vo), and AllegroGraph (ag), work with Goo.
See details and benchmarks in ontoportal-lirmm#48

vemonet and others added 30 commits April 13, 2017 17:24
… a value only the object with a language included in main_languages will be taken (value with no lang defined are taken too). Be careful if an ontology contains several prefLabel with lang defined here
The main_lang is the main lang used to extract prefLabels. Accepted lang is used to extract all others properties
…bo_merge_goo_into_virtual-appliance-3.0-merge
syphax-bouazzouni and others added 23 commits December 5, 2023 11:55
…es (#35)

* add tests for the new dsl to write property data types

* append the property :type values to the :enforce array

* update solution mapper to support multilingual

* update solution mapper to support multilingual

* fix typo ( name )

* add validators tests file

* add validator interface module

* implement data_type validator

* migrate existence validator to the new DSL

* migrate uniqueness validator to the new DSL

* implement object_type validator with the new DSL

* migrate range validator to the new DSL

* refactor the enforce module to use the new validators implementation

* force to regenerate the id when we update related attribute (named_with)

* require the validators implementation

* update existence validator to not accept empty to_s  objects

* update exist? test

* add symmetric validator tests for no_list and list cases

* implement symmetric validator

* move re used methods to the parent class

* update symmetric code and error message

* add distinct of validator tests

* implement distinct_of validator

* add superior_equal_to validator tests

* extract property method to ValidatorBase class

* implement superior_equal_to validator

* add inverse of validator tests

* implement inverse_of validator

* use the class method property in distinct of

* add proc validator tests

* add instance proc validators

* fix call_proc validator to test if the returned values are correct

* add model_with_yaml_scheme test

* implement YAMLScheme module

* use YAMLScheme module in Settings module

* use platform lang and code refacto

* filter by lang in properties

* do some refactoring

* add unmapped_get to goo resources

* update lang filter module to support requested_lang and portal_lang

* use the new lang filter module in the solution_mapper

* remove the usage of the old lang filter module in map_attributes

* add request language global variable

* fix datatype check for list values

* remove old unused test if clause

* for no unmapped values cast them to object before sending

* for resource unmapped_set merge new value if an array

* prevent add_object_to_model if no_lang and previous value exist

* move from the mapper  lang_filter related code to  lang_filter module

* move internal lang filter module methods to private section

* add request_store gem to save request language globally

* save requested  language in model_load options

* force requested_lang and portal_langs to be upcase and symbol

* change methodes/vars names

* get the last item in objects instead of passing the current object

* Revert "get the last item in objects instead of passing the current object"

This reverts commit 996922a.

* handle this case where values is nil in save_model_values

* handle the casf of nil values for  the SuperiorEqualTo validator

* add onUpdate callback tests

* implement enforce_callback to run an attribute callback

* move the attribute default callback to the save method

* implement onUpdate DSL in the ressource settings

* call to the attributes onUpdate callback in the save method

* in validators bring attribute if needed

* make superior_equal_to works for list attributes

* add email validator test

* implement email validator

* add filters patterns to select variables

* make make regex filter no-case sensitive

* if requested_lang = 'all' return all

* support select multilanguage

* show the values with their corresponding language

* use @attributes_to_translate

* change methode name

* remove platform languages

* add complex_order_by unit test

* refactor query_builder to extract internal_variables as instance variable

* update order_by to work for joined patterns (object attributes)

* downcase lang key

* Fix the issue of undefined 'id'  of the language filter module

* Show literal attribute if we requested all the languages

* Use portal language by default in the language filter module

* group unmapped properties by lang

* Feature: group unmapped properties by language  (#38)

* group unmapped properties by lang

* downcase language keys of unmapped properties

---------

Co-authored-by: Syphax bouazzouni <gs_bouazzouni@esi.dz>

* assert that pre in an array in get_value_object

* add label to attributes_to_translate

* update define_method

* update solution mapper

* update get_preload_value

* Feature: Support multi lingual -  add show_language argument to the attributes getters (#39)

* update define_method

* update solution mapper

* update get_preload_value

* fix save_model_values if unmmaped condition

* fix getters for list attributes  to not take only the first value

* remove the languages hash for the unmapped if not a mutli langual asked

* move some language helper from the mapper to the lang_filter module

* move @requested_lang variable from the mapper to the lang_filter module

* remove no more used @attributes_to_translate variable in lang_filter

* fix save_model_values method to not save RDF:Literal object but a string

* remove  not used method in lang filter module

* refecator and rename some methods of the lang_filter module

* use the new name of the lang filter methods in the solution mapper

* replace the getters argument to show languages from :show_all_languages to :show_languages: true

* catch transform_values of unmapped if it is nil

* change the getters show_all_languages argument from to include_languages

* make the map_attributes handle the option showing all the languages

* fix order by an attribute that is already filtered

* don't add the filtered variables to the select clause of the query

* add filters patterns to select variables

* fix pagination with order_by with filter that returns empty pages for 4store

* include the in the select variables filtered variables

* optimize pagination query by not re-doing the filters and order in the include query

---------

Co-authored-by: HADDAD Zineddine <hz_haddad@esi.dz>
@syphax-bouazzouni syphax-bouazzouni force-pushed the pr/feature/support-multiple-backends branch 3 times, most recently from e7fe15a to 5242514 Compare April 17, 2024 19:42
@syphax-bouazzouni syphax-bouazzouni force-pushed the pr/feature/support-multiple-backends branch from 5242514 to d316639 Compare April 17, 2024 20:04
@syphax-bouazzouni syphax-bouazzouni changed the title Feature: Support multiple backends Feature: Support multiple backends (4store, GraphDb, Virtuoso, and AllegroGraph) Apr 17, 2024
@codecov-commenter
Copy link

Codecov Report

Attention: Patch coverage is 80.14311% with 111 lines in your changes are missing coverage. Please review.

Project coverage is 85.37%. Comparing base (87e991b) to head (d316639).
Report is 29 commits behind head on develop.

Files Patch % Lines
lib/goo/sparql/mixins/solution_lang_filter.rb 58.42% 37 Missing ⚠️
lib/goo/sparql/processor.rb 74.39% 21 Missing ⚠️
lib/goo/sparql/query_builder.rb 86.31% 13 Missing ⚠️
lib/goo/sparql/solutions_mapper.rb 86.74% 11 Missing ⚠️
lib/goo.rb 66.66% 8 Missing ⚠️
lib/goo/base/resource.rb 89.74% 8 Missing ⚠️
lib/goo/base/settings/settings.rb 88.88% 4 Missing ⚠️
lib/goo/search/search.rb 20.00% 4 Missing ⚠️
lib/goo/sparql/client.rb 78.57% 3 Missing ⚠️
lib/goo/validators/enforce.rb 77.77% 2 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #152      +/-   ##
===========================================
+ Coverage    84.53%   85.37%   +0.83%     
===========================================
  Files           35       38       +3     
  Lines         2516     2700     +184     
===========================================
+ Hits          2127     2305     +178     
- Misses         389      395       +6     
Flag Coverage Δ
unittests 85.37% <80.14%> (+0.83%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants