You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
To decrease costs, many Brazilian counties publish in gazettes in groups, like the Alagoas' County Association (AMA). Those gazettes (example here) are distinct from what QD deals, so far: they are one file which contains executive information about many counties. This is no easy task, so OKBR and IFAL joined forces to tackle this problem in the context of the Alagoas state, which counts 102 cities. That resulted in a project financed by IFAL and supported by OKBR, where you can follow the progress here.
So, this issue is about changing the data processing pipeline to use this code to split the SIGPUb gazette and to store each fragment as a different entry. After discussing with @giuliocc, we outlined a few decisions:
must not change the querido-diario, which might make onboarding harder
To decrease costs, many Brazilian counties publish in gazettes in groups, like the Alagoas' County Association (AMA). Those gazettes (example here) are distinct from what QD deals, so far: they are one file which contains executive information about many counties. This is no easy task, so OKBR and IFAL joined forces to tackle this problem in the context of the Alagoas state, which counts 102 cities. That resulted in a project financed by IFAL and supported by OKBR, where you can follow the progress here.
The spider was ready and fetches gazette created according to the Sistema Gerenciador de Publicações Legais (SIGPub). The code to split the SIGPub gazette's text into each county's content is ready to be used in the context of Alagoas. Even though there is no hard restriction, all the 40+ automated tests created so far used gazettes from Alagoas.
So, this issue is about changing the data processing pipeline to use this code to split the SIGPUb gazette and to store each fragment as a different entry. After discussing with @giuliocc, we outlined a few decisions:
if gazette is in this list, do that.
sub-gazettes
will be a compound of the main one with some county id@giuliocc already set up a test environment and gave me access to it. Thanks a lot!
cc/ @alex-custodio @Luisa-Coelho
The text was updated successfully, but these errors were encountered: