Corrige download XLS/CSV da listagem de periódicos no SciELO.org - #1475
Merged
samuelveigarangel merged 5 commits intoSep 15, 2026
Merged
samuelveigarangel merged 5 commits into
samuelveigarangel merged 5 commits into
Conversation
A montagem de scielo_url, publisher e o dicionário do download passa a viver no modelo, para a view só consultar e serializar. Co-authored-by: Cursor <cursoragent@cursor.com>
O arquivo deixa de sair só com o cabeçalho, aplica os mesmos filtros da listagem e nomeia o download com a categoria ou all_journals. Co-authored-by: Cursor <cursoragent@cursor.com>
A listagem por área passa a voltar para todos os periódicos e o botão de download respeita a categoria da URL. Co-authored-by: Cursor <cursoragent@cursor.com>
Os testes garantem que o XLS/CSV traz periódicos, filtra por área e usa all_journals ou o slug da categoria no filename. Co-authored-by: Cursor <cursoragent@cursor.com>
Comment on lines
+2239
to
+2248
| def scielo_url(self): | ||
| domain = "" | ||
| if self.collection and self.collection.domain: | ||
| domain = self.collection.domain | ||
| if not domain: | ||
| return "" | ||
| return ( | ||
| f"{domain.rstrip('/')}/scielo.php?script=sci_serial" | ||
| f"&pid={self.issn_scielo or ''}&lng=en" | ||
| ) |
Contributor
There was a problem hiding this comment.
Uma dúvida:
Essa URL é a padrão para coleção clássica. Pelo que notei, ao acessar para periódico brasileira, ela redireciona para o site opac. Ou seja, https://www.scielo.br/scielo.php?script=sci_serial&pid=2318-8219&lng=es&nrm=iso é redirecionado para https://www.scielo.br/j/abcic/. Pretende-se implementar algo aqui para suportar diretamente o domínio do opac (que usa acronym do periódico) e não precisar do redirect?
Collaborator
Author
There was a problem hiding this comment.
Pelo o que me foi informado, apenas a coleção do brasil e republica dominicana aderiu ao novo site, e ambos possuem redirecionamento.
pitangainnovare
requested changes
Sep 13, 2026
pitangainnovare
left a comment
pitangainnovare
left a comment
Contributor
There was a problem hiding this comment.
- Ha dois testes simples quebrados.
- Há uma dúvida em relação ao campo owner: o cabeçalho do CSV mostra publisher, mas as linhas são preenchidas com owner; pelo que vi, isso já era antes do PR. Seria o caso de alterar para outro campo ou é assim mesmo?
- Há um export silencioso de todos os journals quando a categoria informada é inválida. Seria o caso de gerar algum erro ou manter assim mesmo?
Essencialmente, basta corrigir os testes. Os outros pontos são mais para discussão.
| filename = f"journals_{date}.xls" | ||
| category = request.GET.get("category") if request is not None else None | ||
| if category and slugs_to_category_code.get(category): | ||
| prefix = f"{category.replace('-', '_')}_journals" |
Contributor
There was a problem hiding this comment.
esse _journals causa uma falha nos testes:
- test_csv_download_filters_by_category
- test_xls_download_filename_uses_category_when_present
(base) rafaeljpd@feeradesk-x3d:~/Repos/pi/scl/core$ docker compose -f local.yml run --rm django pytest core/home/tests.py -q
WARN[0000] /home/rafaeljpd/Repos/pi/scl/core/local.yml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion
[+] Creating 3/3
✔ Container scielo_core_local_mailhog Running 0.0s
✔ Container scielo_core_local_postgres Running 0.0s
✔ Container scielo_core_local_redis Running 0.0s
[+] Running 1/1
✔ Container core-solr-1 Started 0.0s
PostgreSQL is available
Test session starts (platform: linux, Python 3.11.13, pytest 7.4.3, pytest-sugar 0.9.7)
django: version: 5.2.7, settings: config.settings.test (from option)
rootdir: /app
configfile: pytest.ini
plugins: Faker-40.38.0, sugar-0.9.7, django-test-migrations-1.3.0, django-4.8.0, anyio-3.7.1
core/home/tests.py ✓✓✓✓ 31% ███▏
―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― TestGetScieloJournalsData.test_csv_download_filters_by_category ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
self = <core.home.tests.TestGetScieloJournalsData testMethod=test_csv_download_filters_by_category>
def test_csv_download_filters_by_category(self):
health = Subject.objects.create(
creator=self.user,
code="Health Sciences",
value="Health Sciences",
)
self.journal.subject.add(health)
other_journal = Journal.objects.create(
creator=self.user,
title="Other Journal",
)
SciELOJournal.objects.create(
issn_scielo="3333-3333",
collection=self.collection,
journal=other_journal,
journal_acron="tjother",
status="C",
)
response = self.client.get(
reverse("download_csv_journals_page_scielo_org"),
{"category": "health-sciences"},
)
content = response.content.decode("utf-8")
self.assertIn("Test Journal Peru", content)
self.assertNotIn("Other Journal", content)
date = timezone.now().strftime("%Y-%m-%d")
> self.assertIn(
f'filename="health-sciences_{date}.csv"',
response["Content-Disposition"],
)
E AssertionError: 'filename="health-sciences_2026-09-13.csv"' not found in 'attachment; filename="health_sciences_journals_2026-09-13.csv"'
core/home/tests.py:182: AssertionError
--------------------------------------------------------------------------------------------------- Captured stderr call ----------------------------------------------------------------------------------------------------
INFO 2026-09-13 14:32:20,335 views 1 134249559516992 Generated CSV file with: 1 journals
----------------------------------------------------------------------------------------------------- Captured log call -----------------------------------------------------------------------------------------------------
INFO core.home.views:views.py:156 Generated CSV file with: 1 journals
core/home/tests.py ⨯✓✓✓✓✓✓✓ 92% █████████▎
―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― TestGetScieloJournalsData.test_xls_download_filename_uses_category_when_present ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
self = <core.home.tests.TestGetScieloJournalsData testMethod=test_xls_download_filename_uses_category_when_present>
def test_xls_download_filename_uses_category_when_present(self):
response = self.client.get(
reverse("download_xls_journals_page_scielo_org"),
{"category": "health-sciences"},
)
date = timezone.now().strftime("%Y-%m-%d")
> self.assertIn(
f'filename="health-sciences_{date}.xls"',
response["Content-Disposition"],
)
E AssertionError: 'filename="health-sciences_2026-09-13.xls"' not found in 'attachment; filename="health_sciences_journals_2026-09-13.xls"'
core/home/tests.py:201: AssertionError
--------------------------------------------------------------------------------------------------- Captured stderr call ----------------------------------------------------------------------------------------------------
INFO 2026-09-13 14:32:20,439 views 1 134249559516992 Generated XLS file with: 0 journals
----------------------------------------------------------------------------------------------------- Captured log call -----------------------------------------------------------------------------------------------------
INFO core.home.views:views.py:132 Generated XLS file with: 0 journals
core/home/tests.py ⨯ 100% ██████████
===================================================================================================== warnings summary ======================================================================================================
../usr/local/lib/python3.11/site-packages/django/db/backends/utils.py:98
/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py:98: RuntimeWarning: Accessing the database during app initialization is discouraged. To fix this warning, avoid executing queries in AppConfig.ready() or when your app modules are imported.
warnings.warn(self.APPS_NOT_READY_WARNING_MSG, category=RuntimeWarning)
core/home/tests.py::TestGetScieloJournalsData::test_csv_download_contains_journal_rows
core/home/tests.py::TestGetScieloJournalsData::test_csv_download_filename_uses_all_journals_without_category
core/home/tests.py::TestGetScieloJournalsData::test_csv_download_filters_by_category
core/home/tests.py::TestGetScieloJournalsData::test_xls_download_contains_journal_rows
core/home/tests.py::TestGetScieloJournalsData::test_xls_download_filename_uses_category_when_present
/usr/local/lib/python3.11/site-packages/django/core/handlers/base.py:61: UserWarning: No directory at: /app/staticfiles/
mw_instance = middleware(adapted_handler)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================================================================================== short test summary info ==================================================================================================
FAILED core/home/tests.py::TestGetScieloJournalsData::test_csv_download_filters_by_category - AssertionError: 'filename="health-sciences_2026-09-13.csv"' not found in 'attachment; filename="health_sciences_journals_2026-09-13.csv"'
FAILED core/home/tests.py::TestGetScieloJournalsData::test_xls_download_filename_uses_category_when_present - AssertionError: 'filename="health-sciences_2026-09-13.xls"' not found in 'attachment; filename="health_sciences_journals_2026-09-13.xls"'
Results (1.16s):
11 passed
2 failed
- core/home/tests.py:155 TestGetScieloJournalsData.test_csv_download_filters_by_category
- core/home/tests.py:195 TestGetScieloJournalsData.test_xls_download_filename_uses_category_when_present
(base) rafaeljpd@feeradesk-x3d:~/Repos/pi/scl/core$ | return { | ||
| "title": title, | ||
| "scielo_url": self.scielo_url, | ||
| "owner": self.owner_name, |
Contributor
There was a problem hiding this comment.
A terceira coluna do arquivo é apresentada como publisher, mas ela é preenchida com owner_name. Vi que isso já era assim.
Seria o caso de alterar para o campo publisher (publisher_names ou algo assim)?
Comment on lines
+75
to
+78
| if category: | ||
| category_code = slugs_to_category_code.get(category) | ||
| if category_code: | ||
| filters &= Q(journal__subject__code=category_code) |
Contributor
There was a problem hiding this comment.
Se informar uma categoria inexistente, todos os journals são retornados.
Nada bloqueante.
Collaborator
Author
There was a problem hiding this comment.
Boa questão. Eu acho que não vejo problema em retornar todos os journals. O que você acha ?
… test_csv_download_filters_by_category
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
O que esse PR faz?
Corrige o download XLS/CSV da listagem de periódicos no SciELO.org, que saía apenas com o nome das colunas. O arquivo passa a incluir os periódicos visíveis na listagem, filtrar por categoria quando o download parte da URL de uma área, e nomear o arquivo com o slug da categoria ou
all_journals. Também adiciona a opção Todos no select de categoria.Onde a revisão poderia começar?
core/home/views.py(_get_scielo_journals_datae_journals_download_filename) e, em seguida,journal/models.py(SciELOJournal.scielo_url,owner_nameeas_export_dict).Como este poderia ser testado manualmente?
all_journals_AAAA-MM-DD.csv(ou.xls).lista-por-tema/health-sciences/) e baixar XLS/CSV: o arquivo deve conter só os periódicos daquela área e o nomehealth-sciences_AAAA-MM-DD.csv.Algum cenário de contexto que queira dar?
A view engolia qualquer exceção ao montar a URL (
domain.rstripcomdomainnulo) e devolvia lista vazia, por isso o download vinha só com o cabeçalho. A listagem por categoria já filtrava porjournal__subject__code, mas o botão de download não enviava a categoria.Screenshots
Não aplicável.
Quais são os tickets relevantes?
Não há issue vinculada.
Referências
Não há.
Segurança da informação (NSI.04)
Este PR manipula dados sensíveis ou pessoais (LGPD)?
Este PR altera autenticação, autorização, controle de acesso ou gerenciamento de sessão?
Este PR introduz, atualiza ou remove dependências de terceiros?
Este PR foi validado pelo pipeline de segurança (SonarQube / Trivy)?
Este PR concatena, monta ou executa comandos SQL, HTML ou JavaScript a partir de entrada externa?
Este PR expõe novos endpoints, telas ou serviços?
Algum segredo, senha, chave ou token está sendo adicionado ao código-fonte?
Made with Cursor