test(ccli): add fixture corpus for CCLI paste parser

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
This commit is contained in:
Thorsten Bus 2026-05-10 18:24:29 +02:00
parent a10068e783
commit 02de6b03c0
27 changed files with 637 additions and 0 deletions

View file

@ -0,0 +1,8 @@
ls tests/fixtures/ccli/*.txt | wc -l
22
grep -rl "Strophe\|Refrain" tests/fixtures/ccli/ | wc -l
4
grep -rl "(Repeat)" tests/fixtures/ccli/ | wc -l
2

View file

@ -0,0 +1,9 @@
ddev exec php artisan test --filter=CcliFixtureSanityTest
PASS Tests\Feature\CcliFixtureSanityTest
✓ ccli fixture corpus has at least 20 txt files
✓ each ccli fixture is valid utf8 with section labels and title
✓ ccli fixture corpus covers german labels
✓ ccli fixture corpus covers repeat markers
Tests: 4 passed (160 assertions)

View file

@ -0,0 +1,61 @@
# CCLI SongSelect Import — Learnings
## [2026-05-10] Session Start
### Architecture Decisions
- Manual paste + bookmarklet approach (NO server-side scraping — Cloudflare/ToS blocker)
- CcliPasteParser is closure-injectable (mirrors ChurchToolsService pattern for testability)
- All songs upserted via CcliImportService mirroring ProImportService::import() shape
- Translation stored inline on SongSlide.text_content_translated (no separate model)
- default_translation_language = APP-GLOBAL Setting (not per-user)
### Key Codebase Facts
- Song.ccli_id is UNIQUE indexed nullable — primary CCLI match key
- SongSlide: text_content (original), text_content_translated (translation)
- Labels are GLOBAL (shared across all songs) — labels table with name + color
- ProImportService::import() is the template for upsert (not upsertSong — that method doesn't exist)
- SettingsController::AGENDA_KEYS constant whitelist for Settings KV
- TranslationService::importFromText distributes lines preserving local slide line counts
- ArrangementDialog.vue lines 488-532 = searchable song select (where CCLI buttons go)
### CCLI SongSelect "View Lyrics" Page Format
- Title on first non-empty line
- Section label as standalone line (e.g., "Verse 1", "Chorus")
- Lyrics lines under each section
- Footer: copyright (©), CCLI number (e.g., "CCLI # 1234567"), author
### Section Label Regex (English + German + variants)
```
/^(Verse|Chorus|Bridge|Pre-Chorus|Tag|Ending|Intro|Interlude|Outro|Misc|Strophe|Refrain|Brücke|Vorrefrain|Schluss|Zwischenspiel)\s*(\d+[a-z]?)?(\s*\((?:Repeat|Wdh\.?)\))?(\s*[xX]\s*\d+)?$/i
```
### Language Mapping (EN ↔ DE)
- Verse ↔ Strophe
- Chorus ↔ Refrain
- Bridge ↔ Brücke
- Pre-Chorus ↔ Vorrefrain
- Ending ↔ Schluss
- Interlude ↔ Zwischenspiel
### Test Fixture Format
Fixtures are synthetic CCLI-format text files. Format:
```
Test Song Title
Test Artist
Verse 1
Line 1 of verse
Line 2 of verse
Chorus
Chorus line 1
Chorus line 2
© 2024 Test Publishing
CCLI # 9999001
```
### Fixture Corpus Notes
- Keep fixture titles/artists anonymized and numeric (`Test Song N`, `Test Artist N`)
- Include both English and German section labels in the corpus so parser regex coverage stays broad
- Add edge cases for missing footer pieces, whitespace, repeat markers, and suffix labels (`2a`, `x2`, `(Repeat)`)

View file

@ -0,0 +1,66 @@
<?php
use Illuminate\Support\Facades\File;
test('ccli fixture corpus has at least 20 txt files', function (): void {
$files = glob(base_path('tests/fixtures/ccli/*.txt'));
expect($files)->not->toBeEmpty();
expect(count($files))->toBeGreaterThanOrEqual(20);
});
test('each ccli fixture is valid utf8 with section labels and title', function (): void {
$files = glob(base_path('tests/fixtures/ccli/*.txt'));
expect($files)->not->toBeEmpty();
foreach ($files as $file) {
$content = File::get($file);
$basename = basename($file);
expect(mb_detect_encoding($content, 'UTF-8', true))->toBe('UTF-8', $basename.' not UTF-8');
expect(strlen($content))->toBeGreaterThanOrEqual(100, $basename.' too small');
expect(strlen($content))->toBeLessThanOrEqual(50000, $basename.' too large');
expect((bool) preg_match('/^(Verse|Chorus|Bridge|Pre-Chorus|Tag|Ending|Intro|Interlude|Outro|Misc|Strophe|Refrain|Brücke|Vorrefrain|Schluss|Zwischenspiel)/im', $content))->toBeTrue($basename.' has no section label');
$lines = preg_split('/\r\n|\n|\r/', $content);
$firstNonEmpty = '';
foreach ($lines as $line) {
if (trim($line) !== '') {
$firstNonEmpty = trim($line);
break;
}
}
expect($firstNonEmpty)->not->toBeEmpty($basename.' has no title line');
}
});
test('ccli fixture corpus covers german labels', function (): void {
$files = glob(base_path('tests/fixtures/ccli/*.txt'));
$germanLabelFound = false;
foreach ($files as $file) {
if (preg_match('/Strophe|Refrain|Brücke/i', File::get($file))) {
$germanLabelFound = true;
break;
}
}
expect($germanLabelFound)->toBeTrue('No fixture with German labels found');
});
test('ccli fixture corpus covers repeat markers', function (): void {
$files = glob(base_path('tests/fixtures/ccli/*.txt'));
$repeatFound = false;
foreach ($files as $file) {
if (preg_match('/\(Repeat\)|[xX]\s*\d+/i', File::get($file))) {
$repeatFound = true;
break;
}
}
expect($repeatFound)->toBeTrue('No fixture with repeat markers found');
});

29
tests/fixtures/ccli/5-verses.txt vendored Normal file
View file

@ -0,0 +1,29 @@
Test Song 22
Test Artist 22
Verse 1
First verse opens with a hopeful line
Keeping time with the steady rhyme
Verse 2
Second verse moves the story forward
Singing truth with a gentler chord
Verse 3
Third verse carries the middle ground
Bringing peace in a calming sound
Verse 4
Fourth verse points us to the light
Holding fast through the longest night
Verse 5
Fifth verse closes the journey well
With a final line that rings and swells
Chorus
We will sing until the dawn
Faithful hearts will carry on
© 2024 Test Publishing House
CCLI # 9999022

50
tests/fixtures/ccli/README.md vendored Normal file
View file

@ -0,0 +1,50 @@
# CCLI Fixture Corpus
Synthetic CCLI SongSelect "View Lyrics" page text fixtures for testing the `CcliPasteParser`.
## Anonymization
- Song titles use `Test Song N`
- Artist names use `Test Artist N`
- CCLI IDs use `9999XXX`
- Lyrics are synthetic but keep realistic SongSelect structure
## Format
Each fixture follows this structure:
1. Song title (first line)
2. Artist name
3. Blank line
4. Section label on its own line
5. Lyric lines
6. Repeated section blocks as needed
7. Footer copyright line starting with `©`
8. Footer `CCLI # NNNNNNN`
## Coverage
| File | Purpose |
|------|---------|
| english-only-multi-verse.txt | Standard EN song with Verse/Chorus/Bridge |
| english-only-single-verse.txt | Minimal song with one verse and one chorus |
| english-german-side-by-side.txt | Alternating EN/DE labels |
| english-french.txt | English labels with French lyrics |
| english-spanish.txt | English labels with Spanish lyrics |
| english-dutch.txt | English labels with Dutch lyrics |
| english-italian.txt | English labels with Italian lyrics |
| german-only.txt | German labels only |
| repeat-marker.txt | Repeat markers like `(Repeat)` and `x2` |
| verse-letter-suffix.txt | Verse suffixes like `2a` and `2b` |
| mixed-german-english-labels.txt | Mixed EN/DE labels |
| missing-copyright.txt | No copyright line |
| missing-year.txt | `©` without year |
| whitespace-edge-cases.txt | Leading/trailing spaces and double blanks |
| umlauts.txt | German umlauts and ß |
| long-bridge.txt | Long bridge with internal blank line |
| pre-chorus.txt | Pre-Chorus sections |
| tag-ending.txt | Tag and Ending sections |
| intro-outro.txt | Intro and Outro sections |
| interlude-misc.txt | Interlude and Misc sections |
| no-translation.txt | English-only import case |
| 5-verses.txt | Stress test with five verses |

21
tests/fixtures/ccli/english-dutch.txt vendored Normal file
View file

@ -0,0 +1,21 @@
Test Song 6
Test Artist 6
Verse 1
In het licht van Uw genade
We find our place and stand secure
Chorus
We praise Your name with lifted hands
Uw trouw blijft voor altijd bestaan
Verse 2
Wanneer de stormen om ons slaan
You calm the sea and carry us through
Chorus
We praise Your name with lifted hands
Uw trouw blijft voor altijd bestaan
© 2024 Test Publishing House
CCLI # 9999006

21
tests/fixtures/ccli/english-french.txt vendored Normal file
View file

@ -0,0 +1,21 @@
Test Song 4
Test Artist 4
Verse 1
Nous levons les yeux vers toi Seigneur
Ta paix remplit nos cœurs ce matin
Chorus
We sing Your name with joyful sound
Ton amour nous porte maintenant
Verse 2
Dans le silence, ta voix nous guide
Every step is safe within Your light
Chorus
We sing Your name with joyful sound
Ton amour nous porte maintenant
© 2024 Test Publishing House
CCLI # 9999004

View file

@ -0,0 +1,21 @@
Test Song 3
Test Artist 3
Verse 1
English lyrics line 1 for the opening theme
English lyrics line 2 keeps the melody warm
Strophe 1
Deutsche Liedzeile 1 zum gleichen Gedanken
Deutsche Liedzeile 2 trägt den Refrain vor
Chorus
English chorus line 1 with bright harmony
English chorus line 2 with steady praise
Refrain
Deutscher Refrain Zeile 1 für die Gemeinde
Deutscher Refrain Zeile 2 klingt gemeinsam
© 2024 Test Publishing House
CCLI # 9999003

21
tests/fixtures/ccli/english-italian.txt vendored Normal file
View file

@ -0,0 +1,21 @@
Test Song 7
Test Artist 7
Verse 1
Nel silenzio ascoltiamo Te
We are renewed by mercy's song
Chorus
We will follow where You lead
Ti adoriamo con verità
Verse 2
Ogni passo porta pace nuova
Your light remains and never fades
Chorus
We will follow where You lead
Ti adoriamo con verità
© 2024 Test Publishing House
CCLI # 9999007

View file

@ -0,0 +1,26 @@
Test Song 1
Test Artist 1
Verse 1
Morning light breaks through the quiet room
Hands are open, hearts awake to You
Grace is rising like the dawn anew
Chorus
We lift our song and breathe Your name
We lift our eyes and sing Your praise
Verse 2
When the night has tried to claim our hope
You remain the anchor of our souls
Bridge
Faith will stand when every fear has passed
Love will last, Your promise holds fast
Chorus
We lift our song and breathe Your name
We lift our eyes and sing Your praise
© 2024 Test Publishing House
CCLI # 9999001

View file

@ -0,0 +1,13 @@
Test Song 2
Test Artist 2
Verse 1
You are near in every breath we take
You are here when shadows start to fade
Chorus
Jesus, You are steady and true
We rest our hope and trust in You
© 2024 Test Publishing House
CCLI # 9999002

21
tests/fixtures/ccli/english-spanish.txt vendored Normal file
View file

@ -0,0 +1,21 @@
Test Song 5
Test Artist 5
Verse 1
Tu gracia llena cada rincón
We stand amazed before Your throne
Chorus
We will declare Your faithful love
Cantaremos hoy tu gran verdad
Verse 2
Cuando el camino no se ve
Your hand is still the one we seek
Chorus
We will declare Your faithful love
Cantaremos hoy tu gran verdad
© 2024 Test Publishing House
CCLI # 9999005

21
tests/fixtures/ccli/german-only.txt vendored Normal file
View file

@ -0,0 +1,21 @@
Test Song 8
Test Artist 8
Strophe 1
Du führst uns sanft durch jeden Tag
Dein Licht bleibt nah, auch wenn es dunkel mag
Refrain
Wir singen laut von deiner Gnade
Dein Name bleibt, was immer komme
Strophe 2
Wenn Zweifel kommen, hältst du fest
Dein Friede gibt uns neuen Rest
Brücke
Du bist treu, du bist treu, du bist treu
Unser Herz vertraut dir neu
© 2024 Test Publishing House
CCLI # 9999008

20
tests/fixtures/ccli/interlude-misc.txt vendored Normal file
View file

@ -0,0 +1,20 @@
Test Song 20
Test Artist 20
Verse 1
We enter in with open hands
Trusting You will lead the band
Interlude
Instrumental line carries the thought
Misc
Spoken blessing for the room
Quiet hope that fills the tomb
Chorus
We declare Your name aloud
Standing humble, strong, and proud
© 2024 Test Publishing House
CCLI # 9999020

20
tests/fixtures/ccli/intro-outro.txt vendored Normal file
View file

@ -0,0 +1,20 @@
Test Song 19
Test Artist 19
Intro
Instrumental opening for the gathering
Verse 1
You call us close and make us whole
You hold the broken, heal the soul
Chorus
We worship You with all we are
You shine like morning, near and far
Outro
Soft and steady as we go
Your peace remains and overflows
© 2024 Test Publishing House
CCLI # 9999019

23
tests/fixtures/ccli/long-bridge.txt vendored Normal file
View file

@ -0,0 +1,23 @@
Test Song 16
Test Artist 16
Verse 1
We start in silence and receive Your peace
Every burden falls as Your kindness speaks
Bridge
When the night is long, You are still our song
We can trust Your word; it has never failed
There is room for hope in the waiting line
Even now Your light reaches every heart
We will not give up, we will not let go
You are with us here, and You make us whole
Chorus
We lift our hands and sing again
Your faithful love will never end
© 2024 Test Publishing House
CCLI # 9999016

View file

@ -0,0 +1,12 @@
Test Song 12
Test Artist 12
Verse 1
Simple lines for a missing footer test
The parser should still read the body
Chorus
We are here to sing again
Lifted hope in words and men
CCLI # 9999012

13
tests/fixtures/ccli/missing-year.txt vendored Normal file
View file

@ -0,0 +1,13 @@
Test Song 13
Test Artist 13
Verse 1
This fixture keeps the symbol only
No year is written in the footer line
Chorus
We still can see the copyright mark
And parse the lines in steady order
© Test Publishing House
CCLI # 9999013

View file

@ -0,0 +1,21 @@
Test Song 11
Test Artist 11
Strophe 1
Unsere Herzen stehen auf
Your mercy lifts us higher now
Chorus
We give You praise forevermore
Gemeinsam singen wir empor
Strophe 2
Wenn der Abend still wird hier
We know You stay, You are near
Refrain
Dein Name klingt in jedem Raum
We follow You and hold the sound
© 2024 Test Publishing House
CCLI # 9999011

21
tests/fixtures/ccli/no-translation.txt vendored Normal file
View file

@ -0,0 +1,21 @@
Test Song 21
Test Artist 21
Verse 1
This song is plain and purely English
It gives a simple import test case
Chorus
We sing the words as written here
No translation layer appears
Verse 2
Every line remains intact
For parser coverage and feedback
Chorus
We sing the words as written here
No translation layer appears
© 2024 Test Publishing House
CCLI # 9999021

29
tests/fixtures/ccli/pre-chorus.txt vendored Normal file
View file

@ -0,0 +1,29 @@
Test Song 17
Test Artist 17
Verse 1
We walk by faith and not by sight
You lead us through the quiet night
Pre-Chorus
Now our hearts are turning near
Every promise becomes clear
Chorus
We sing because You are enough
We sing because Your love is strong
Verse 2
When the road bends, You are there
Holding every answered prayer
Pre-Chorus
Now our hearts are turning near
Every promise becomes clear
Chorus
We sing because You are enough
We sing because Your love is strong
© 2024 Test Publishing House
CCLI # 9999017

17
tests/fixtures/ccli/repeat-marker.txt vendored Normal file
View file

@ -0,0 +1,17 @@
Test Song 9
Test Artist 9
Verse 1
We remember all Your kindness here
Every step is marked by perfect care
Chorus 1 (Repeat)
Lift the sound again and again
Praise will rise and never end
Bridge x2
Open heaven, pour Your peace
Hold us close and set us free
© 2024 Test Publishing House
CCLI # 9999009

21
tests/fixtures/ccli/tag-ending.txt vendored Normal file
View file

@ -0,0 +1,21 @@
Test Song 18
Test Artist 18
Verse 1
We have seen Your mercy move
We have heard Your steady voice
Chorus
All our days belong to You
We will follow and rejoice
Tag
Holy, holy, holy Lord
We keep singing more and more
Ending
Faithful God, forever stay
Guide our hearts along the way
© 2024 Test Publishing House
CCLI # 9999018

17
tests/fixtures/ccli/umlauts.txt vendored Normal file
View file

@ -0,0 +1,17 @@
Test Song 15
Test Artist 15
Strophe 1
Ähren im Wind, die Herzen singen
Öffne den Himmel, lass Gnade klingen
Refrain
Über allem steht dein Name klar
Fußspuren führen uns Jahr um Jahr
Brücke
Für immer bist du gut und wahr
Größer ist deine Liebe, wunderbar
© 2024 Test Publishing House
CCLI # 9999015

View file

@ -0,0 +1,21 @@
Test Song 10
Test Artist 10
Verse 1
The morning comes and we arise
Hope is rising in our eyes
Verse 2a
When the road is hard and long
You still teach our hearts to sing
Verse 2b
When the answers do not show
Still Your faithfulness will grow
Chorus
We will trust You all our days
And we will walk in steadfast grace
© 2024 Test Publishing House
CCLI # 9999010

View file

@ -0,0 +1,14 @@
Test Song 14
Test Artist 14
Verse 1
Leading spaces on this lyric line
Trailing spaces on this lyric line
Chorus
This chorus follows after double blanks
Another line with extra spaces
© 2024 Test Publishing House
CCLI # 9999014