Repository navigation
feat(importer): read TablEdit (.tef) files - #3013
Open
Danielku15 wants to merge 47 commits into
Open
Danielku15 wants to merge 47 commits into
Danielku15 wants to merge 47 commits into
Conversation
Covers the primary and combination effects, the second-column effects, harmonics, bends to the tied target note, pitch changes, grace notes with their extension records, left and right hand fingerings, pick strokes and ottava attributes.
Reads the text and chord tables and applies text, chord diagram, accent, crescendo, fermata, slur and MIDI patch markers to the beats of their track, plus tempo changes with ramps, syncopation as triplet feel, line breaks and double bars.
…s reading of it Corrections found with files entered in TablEdit: - the instrument bank is the bank MSB - transposing instruments show their written key - chord diagrams take the string count of the instrument they are used on - a reading list with an unknown record size is ignored, and repeat signs are only taken when the file has no reading list - artificial harmonics use the stored harmonic fret, also from the combination byte; natural harmonics get the harmonic value of their fret - grace notes keep their stored fret, also from another string, and a hammer-on grace from another string keeps its slur - trills use the stored trill fret - bends and the tremolo bar take their amount from the next note on the string, tied or not - mordents and turns take their direction from their auxiliary notes - the ghost combination with the ghost effect means no effect - crescendos without length or amount are ignored - accents apply to the notes starting at the marker - gaps after dotted 64ths and incomplete tuplets are filled with 1/128 or tuplet rests - only the first lyrics section of a module is used Additions: - tremolo bar up/down and dive-return as whammy bar - roll and brush speed as brush duration (bits 4-6, 5 TablEdit ticks per step) - excluded-from-beaming notes and secondary beam breaks on rests; free bars are beamed across the bar or its halves - hidden notes and rests where no other shown staff displays them - measures per line from line-break markers - numbered connections as n-tuplets of n : m/2, timed like TablEdit - standalone pick strokes - the let-ring instrument option - chord names in lyrics and lyric text markers - effect parameters stored in the extension record
TablEdit 2.6x and later 2.x versions save .tef files in formats 3 to 9. Their container has nothing in common with the TablEdit 3 one: a fixed header with counts instead of a pointer table, sequential blocks, 6-byte grid records addressed per measure slot, measures of the header's longest size whose shorter variants are right-aligned and announced by a cell, keys in the header or a per-measure block, and partly different cell codes. The records inside (instruments, chords, texts) and the meaning of the note bytes are the ones of TablEdit 3. So TablEdit 2 gets its own container reader, TefFormat2Reader, which translates the measures and the grid into the measure geometry and cell layout the TablEdit 3 path produces. The importer keeps one entry point, detects the format and hands both paths to the shared instrument and chord record decoding, TefBarBuilder and TefReadingList, which stay unchanged. Byte access and text decoding move into TefBinary for both readers. Formats 1 and 2 (TablEdit 1.x and early 2.x) are still rejected with a hint to re-save the file with TablEdit 3. Protected TablEdit 2 files are rejected like protected TablEdit 3 files. Drum and rhythm patterns of TablEdit 2 are skipped with a warning.
TablEdit keeps the abbreviation at byte 26 of the instrument name, introduced by '^' (a name entered as "Lead Guitar^Ld" is stored as "Lead Guitar", leftovers, "^Ld"); the bytes between the end of the name and byte 26 are leftovers of earlier names. A '^' within the name is part of the name, as TablEdit 3 shows it for TablEdit 2 files.
…it importer Import order and line wrapping as the formatter writes them; no change in behaviour.
… jumps Derives repeats with pass counts, alternate endings and D.C./D.S. jumps with coda and fine targets from the reading list, checks the resulting playback order, turns labels into sections, and shows the graphical repeat signs when a file has no reading list. A repeat pass that continues past the repeat (1-2, 1-2, 1-2, 3-n) counts as a pass of the repeat instead of a D.C., and a second D.S. that skips ahead again ends at the double coda (D.S.S. al Double Coda).
…ransposition and drum tabs
…lEdit Files entered in TablEdit 3.06 (and some patched copies of them); each test asserts the entered values per variant: notes, accidentals, ties, dynamics, voices, durations and tuplets, numbered connections, effects and their parameters, grace notes and ornaments, fingering and strokes, attributes, instruments, clefs and drums, time and key signatures, pick-up and flag bits, text, chord, tempo, syncopation and patch markers, line breaks, lyrics and reading lists. The reading list structures and the corrupt file cases patch these files. The tests stay transpilable to C# and Kotlin.
The files in test-data/tabledit/v2 are TablEdit 2 files (format 3) written byte by byte; each <name>.v3.tef is the same file opened in TablEdit 3.06 and saved again. The TablEdit 2 import must equal the import of the conversion, plus explicit checks of notes, effects, markers, measures, instruments, the skipped drum and rhythm patterns and protected files.
…ON, alphaTex and the serializer Visual tests for effects, tuplets, a reading list with voltas and text and chord markers; the TablEdit files also run through the JSON, alphaTex and serializer round trips.
A format page with the feature table of TablEdit 2 and 3 files and their support in alphaTab, a feature page for loading TablEdit files, TablEdit in the lists of supported formats and a release note.
.NET only knows Windows-1252 when the host registers the code pages provider; without it the decoder fell back to UTF-8 and replaced every accented character of a Windows-1252 text.
… tests to the C# and Kotlin runtimes - Uint8Array.fill with a start and end (C#, Kotlin) - a Uint8Array copied from another Uint8Array (Kotlin) - slice with a start and end on lists and boolean lists (Kotlin) - includes and flatMap on number lists (Kotlin) - sort without a comparison returns the sorted list (Kotlin)
…e in JavaScript In the C# and Kotlin test globals, list elements that are numbers or enum values are compared by their numeric value, and Kotlin compares map entries by key and value. C# accepts a message for an expected throwing action, Kotlin has toBeDefined.
Detects TablEdit files, reads title fields, measures, instruments and the note grid, and lays the grid out into alphaTab voices with packed tuplets, lanes for overlapping lengths and rest filling. Triplet runs are spread relative to the beat of their first member, other tuplets relative to their beat, as TablEdit times them. Measure sizes follow the time signature, 12-byte measure records and formats 11 and 12 are recognised, and texts that are not valid UTF-8 are read as Windows-1252.
effects-b5.tef is re-entered in TablEdit with the notes after the two tremolo bars in bar 2 on frets 3 and 7, so the up/down and the dive-return take an amount from the next note on the string. A patched copy keeps the case of a target on the same fret, which has no amount. The format page notes that a tremolo bar without an amount is not shown.
…s and skipped labels A reading list with more entries than TablEdit keeps is now skipped instead of left unread, and a label TablEdit ignores still has its text skipped, so the texts, chords, instruments and labels after them are read from the right place. The crescendo length is stored in a byte as TablEdit stores it, which gives the same length on every platform.
… strings Such files now fail with a FormatError instead of a TypeError, as other corrupt TablEdit data does. A reading-list label on an entry that starts before the first bar is not shown.
…dit 3.04 b6 TablEdit treats the code of such files as 0, so a stale value no longer makes them count as protected.
…mats in one place The cell type moves to its own module with the cell codes and flags that the TablEdit 2 reader and the bar builder share. The importer reads the header fields of either format into the same fields and configures the bar builder once, reads bytes from the file directly, and finds the bar of a reading guide through the bar builder. Values that only restated a default (the crescendo amount of TablEdit 2 files, the zeroed name tail of their instruments, a duplicate stabilo flag, a tuplet number that cannot exceed 15) are gone, and comments that restated the format are removed.
…oss barlines and in linear time The bar builder records the beats of all voices of a bar instead of only the first voice of the track. Markers and lyrics go to the first voice of each bar that has content, so a track with notes in the upper voice keeps the texts, chords, fermatas, crescendos, slurs and lyrics of its other bars. Slurs and crescendos follow their length into the next bars. Cells, events, tuplet ranges and units are grouped by track and bar once, and the durations come from one table, so the import time grows linearly with the number of bars instead of quadratically. A cell without a fret is no note, rests after the last note of a pick-up measure no longer lengthen it, and a tempo ramp that starts in a pick-up measure continues into the next bar. Empty voices are left to the model's consolidation, and checks that the lane assignment already guarantees are gone.
…eader tempo A tempo marker on the first position of the piece set a second tempo next to the header tempo, so two markings were drawn there and the score tempo was not the one that plays.
Grace notes on a drum line became string and fret notes; they now get the drum of their line and fret like the main notes, so flams sound and export.
…06 A4 without the dashed flag These files keep the dashed flag in bit 7 of the length, which TablEdit masks off when it loads them.
…line A '>' inside a lyrics line was taken as the start of a section and cut the lyrics there.
… were found Two units with the same start, kind and length could take each other's voice on platforms whose sort is not stable.
… effects Only the roll speed and the octave of the artificial harmonic differ from the primary effect of the same kind.
…om the C# and Kotlin runtimes The TablEdit 2 reader no longer fills a range of a Uint8Array, and nothing else needs it.
…lEdit These files only had their imports and line wrapping changed by the formatter.
…ly and old protection codes
… more than two items
…untimes The TablEdit tests describe the cells they append as [position, row, bytes].
…files alphaTab reads TablEdit 3 files (formats 10 to 12) only. Files of TablEdit 1 and 2 (formats 1 to 9) are still detected as TablEdit files and rejected with a FormatError asking to open the file in TablEdit 3 and save it again; data that does not look like a TablEdit file still raises an UnsupportedFormatError. TefFormat2Reader, its tests and the TablEdit 2 test files are removed. The importer reads the TablEdit 3 header directly, the instrument and chord records are read from the file again instead of from a translated copy, and TefBinary keeps its range check and text decoding private.
The reading list is now expressed the way alphaTab's playback controller plays it. Playback is normal again after a coda, so going back there becomes a repeat (with endings) or a further D.C./D.S. After a jump a repeat plays its final pass, so skipping its first ending needs no marking; only another skip forward becomes a To Coda and its coda. A D.S. to a bar that already has a segno reuses it, and a pass whose To Coda would stop an earlier pass gets the double coda instead, so no two targets of one kind clash. Calliope Rag (discussion #2963) now plays in TablEdit's order. The reading-list structures gain its list on an empty 49-bar file made in TablEdit, and one case each for a repeat after the coda, a first ending skipped after a jump and a shared segno with separate codas. The format page names the remaining limits: after a jump alphaTab plays every repeat once until the coda.
… bar When a score is read from JSON (e.g. when it is passed to the worker), a note finds the origin of its tie, slur, hammer-on or slide among the notes chained before it. An origin in a voice with a higher index of the same bar is chained later, so the link failed with an error. Such an origin is now looked up in the voices of the bar.
TablEdit ties a note to the previous note on the same string, whatever voice either is in. alphaTab looked for the origin in the voice of the tied note only, so a tie whose origin the importer placed in another voice was dropped and the note struck again, as with the tied chords reported in discussion #2963. The importer now sets the tie origin of every tied note itself. voices-ties.tef was entered in TablEdit with ties from the automatic voice into the upper and lower voice, from the upper into the automatic voice and across a barline; the test checks the origins and that the MIDI strikes only the untied notes. A visual test shows the tie arcs between voices. alphaTab ties a note within its voice in alphaTex, so the alphaTex round trip of this file ignores the tie flag and fret.
alphaTab draws every voice after the first one in the secondary colour. The importer ordered the upper voice first, so as soon as a track used the upper voice anywhere, the automatic voice with most of the notes was drawn grey (discussion #2963). The voices are now ordered automatic, upper, lower per lane. The upper and lower voice keep their stems up and down; the automatic voice turns its stems down in bars with an upper voice. The voice indices in the voices and markers-upper-voice tests follow the new order, and voices-ties checks the stem directions.
TablEdit shows no dynamics markings. The importer hid them only on beats with a struck note, so a beat holding only tied notes showed the default dynamics as a marking.
The coda kind search reused a loop variable name of an enclosing scope, which C# rejects, and the cross-voice tie test described its cases as six-item tuples, which the C# and Kotlin runtimes do not have.
TypeScript infers null as the element type of an array literal holding only nulls. The transpiler had no mapping for the null type, so Kotlin got List<Any>, which cannot hold null. null now maps to the nullable object type like undefined. Fixture: array-of-nulls.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
AI-authored disclosure (
alphatab-ai-authored-v1)Portions of this content were authored by an AI agent. The agent has read
AGENTS.md and the human submitter accepts responsibility for
compliance with the rules in that document.
Issues
Fixes #3006 (requested in discussion #2963)
Proposed changes
alphaTab can now load TablEdit files (
.tef), saved by TablEdit 2.6x and later in the TablEdit 2 (formats 3–9) andTablEdit 3 (formats 10–12) layouts. The new
TablEditImporteris registered in the default importer list, soapi.load,ScoreLoaderand the file pickers of the playground and the UI accept.teffiles without further setup.Why it is built this way. TablEdit does not store beats: it stores notes and markers on a 64th grid per string row. The
importer therefore rebuilds the alphaTab model from that grid instead of mapping records one to one:
TablEditImporterdetects the format, reads the header, measures, instruments, chords, texts and lyrics, and rejectsprotected files (their notes are encrypted) and the oldest formats 1–2 with a hint to re-save in TablEdit 3.
TefBarBuilderturns the grid into voices and beats: notes of different length that sound together go to extra voices,tuplets and numbered connections are timed as TablEdit times them, gaps become rests, and effects, grace notes,
ornaments, fingerings, strokes, markers (text, chord, accent, crescendo, fermata, slur, tempo, syncopation, patch,
line break) and lyrics are attached to the beats of their track.
TefReadingListexpresses TablEdit's reading list (the playback order) through repeats, alternate endings andD.C./D.S./coda/fine jumps, then checks the result against the order alphaTab's playback controller produces and logs a
warning when the order cannot be expressed exactly.
TefFormat2Readertranslates the completely different TablEdit 2 container into the geometry and cell layout ofTablEdit 3, so both formats share the decoding above;
TefBinaryholds the shared byte and text access (UTF-8 with aWindows-1252 fallback for older texts).
Platform runtime. The importer and its tests run in C# and Kotlin too. That needed some runtime additions and fixes
outside the importer, each in its own commit:
fix(kotlin):DoubleList.spread()returned its spare capacity as extra zeros.fix(csharp): Windows-1252 text was decoded as UTF-8 on .NET without a registered code pages provider (this also affectsGuitar Pro 3–5 files read with
importer.encoding = 'windows-1252').fix(platform):Uint8Array.fill(value, start, end)(C#, Kotlin); in Kotlin aUint8Arraycopy constructor,slice(start, end)on lists,includes/flatMapon number lists,sort()returning the list.test(platform): the C# and KotlintoEqualcompare numbers and enum values by value as JavaScript does, Kotlin comparesmap entries by key and value; C# accepts a message for an expected throwing action, Kotlin has
toBeDefined.A separate
style(importer)commit re-formats importer files touched by the formatter; it changes no behaviour.Tests and how the expectations were obtained
test/importer/TablEditImporter.test.ts: 87 tests over 39 files entered in TablEdit 3.06 (plus patched copies forcorrupt-file, reading-list-structure and protection cases). The expected values follow from what was entered in TablEdit
and from the file format, not from importer output: notes, accidentals, ties, dynamics, voices, durations, tuplets and
numbered connections, effects and parameters, grace notes and ornaments, fingering and strokes, instruments, clefs and
drums, time and key signatures, pick-ups, markers, line breaks, lyrics, reading lists (including 16 reading-list
structures checked against TablEdit's playback order) and corrupt or foreign data.
test/importer/TablEdit2Importer.test.ts: 20 tests over 6 TablEdit 2 files written byte by byte; each is compared withits conversion by TablEdit 3.06 (
<name>.v3.tef), which serves as the oracle, plus explicit checks.test/visualTests/features/TablEdit.test.ts: 4 visual tests (effects, tuplets, reading list with voltas, text andchord markers). The reference images are not part of this PR yet — they are under review.
local-only corpus of MuseScore's TablEdit test files (GPL, not added to the repository).
Results on the branch: TypeScript 2512 passed, C# 2372 passed, Kotlin 2372 passed; in each the only failures are the 4
TablEdit visual tests without reference images.
npm run build-websitepasses.Docs. New format page
website/docs/formats/tabledit.mdxwith a 154-row feature table for TablEdit 2 and 3 files(model, reading, rendering, audio, alphaTex), new feature page
features/import-load/tabledit.mdx, TablEdit added to theformat lists (README, introduction, feature pages, landing page, llms.txt, package description) and a release note in
releases/release1_10.mdxwith a demo file.Known limitations (all listed in the feature table):
volume changes; fermatas are not held longer.
hammer-ons from another string are kept as slurs.
courses and special tablature styles, diatonic/lettered tablature (pitches play correctly), thumb/finger strokes, brackets
over notes, scale diagrams, extender lines, beats without stem or flag; notes and rests can only be hidden in all staves.
structure with a warning.
some symbol numbers, beam-break variants, special characters, the rhythm-track option.
(skipped with a warning), TablEdit 2 spacing and grace detail records.
Follow-ups (not in this PR)
Open point before merge: permission from the TablEdit authors to ship the
.teftest files and the website demo file,made with the TablEdit demo, under MPL-2.0 is still pending.
Checklist
AI authorship disclosure
(
alphatab-ai-authored-v1) is present at the top of this body, and I havepersonally reviewed every change and can explain each one
Further details
website/