Skip to content

FIX: Flush pending EIT sections in EPG_free() before freeing buffers#2166

Open
Varadraj75 wants to merge 3 commits intoCCExtractor:masterfrom
Varadraj75:fix/eit-epg-buffer-flush
Open

FIX: Flush pending EIT sections in EPG_free() before freeing buffers#2166
Varadraj75 wants to merge 3 commits intoCCExtractor:masterfrom
Varadraj75:fix/eit-epg-buffer-flush

Conversation

@Varadraj75
Copy link
Contributor

In raising this pull request, I confirm the following (please check boxes):

  • I have read and understood the contributors guide.
  • I have checked that another pull request for this purpose does not exist.
  • I have considered, and confirmed that this submission will be valuable to others.
  • I accept that this submission may not be used, and the pull request closed at the will of the maintainer.
  • I give this submission freely, and claim no ownership to its content.
  • I have mentioned this change in the changelog.

My familiarity with the project is as follows (check one):

  • I have never used CCExtractor.
  • I have used CCExtractor just a couple of times.
  • I absolutely love CCExtractor, but have not contributed previously.
  • I am an active contributor to CCExtractor.

Summary

Fixes #2165 — the last EIT section in any stream was silently discarded.

Root Cause

parse_EPG_packet() accumulates TS packets into epg_buffers[] and only
calls EPG_parse_table() when a new section starts (payload_start_indicator=1).
This means the last accumulated section is never parsed — there is no
following packet to trigger the flush.

EPG_free() then freed the buffers without processing them, silently
discarding the last EIT section in every stream.

Fix

EPG_free() now iterates over all epg_buffers slots before freeing and
flushes any with ccounter > 0:

// Flush any pending EIT sections not triggered by a subsequent
// payload_start_indicator packet (e.g. last section in stream)
for (int i = 0; i <= 0xfff; i++)
{
    if (ctx->epg_buffers[i].buffer != NULL && ctx->epg_buffers[i].ccounter > 0)
    {
        EPG_parse_table(ctx, ctx->epg_buffers[i].buffer, ctx->epg_buffers[i].buffer_length);
        free(ctx->epg_buffers[i].buffer);
        ctx->epg_buffers[i].buffer = NULL;
    }
}

Impact

  • Fixes missing EPG/XMLTV data for the last program entry in streams
  • Particularly impactful for short streams or streams with few EIT sections
    where the last section represents a significant portion of the EPG data

Testing

Built and verified locally on macOS. All existing CI tests pass.

Fixes #2165

parse_EPG_packet() accumulates TS continuation packets into epg_buffers[]
and only calls EPG_parse_table() when a new section starts
(payload_start_indicator=1). This meant the last EIT section in any
stream was never parsed — there is no following packet to trigger the
flush.

EPG_free() now iterates over all epg_buffers slots and flushes any
with ccounter > 0 before freeing, ensuring the last section is always
processed.

Fixes #ISSUE_NUMBER
parse_EPG_packet() accumulates TS continuation packets into epg_buffers[]
and only calls EPG_parse_table() when a new section starts
(payload_start_indicator=1). This meant the last EIT section in any
stream was never parsed — there is no following packet to trigger the
flush.

EPG_free() now iterates over all epg_buffers slots and flushes any
with ccounter > 0 before freeing, ensuring the last section is always
processed.

Fixes CCExtractor#2165
Copilot AI review requested due to automatic review settings March 2, 2026 11:24
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes a long-standing Transport Stream EPG edge case where the final accumulated EIT/PSI section was never parsed because parsing was only triggered when a new section started, causing the last section in the stream to be silently dropped.

Changes:

  • Flush any pending EIT sections from epg_buffers[] during EPG_free() before freeing EPG state.
  • Document the fix in the changelog (references #2165).

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
src/lib_ccx/ts_tables_epg.c Flushes buffered (previously unflushed) EIT sections during cleanup to avoid losing the final section.
docs/CHANGES.TXT Adds a changelog entry describing the fix and linking to the reported issue.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

{
if (ctx->epg_buffers[i].buffer != NULL && ctx->epg_buffers[i].ccounter > 0)
{
EPG_parse_table(ctx, ctx->epg_buffers[i].buffer, ctx->epg_buffers[i].buffer_length);
Copy link

Copilot AI Mar 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before calling EPG_parse_table() here, please validate that the pending buffer is large enough (e.g., buffer_length > 0 and pointer_field+1 < buffer_length) to avoid out-of-bounds reads when the last buffered section is truncated/malformed or when payload_length ended up as 0. This flush path can now parse buffers that previously would have been dropped at shutdown, so it should be resilient to incomplete data.

Suggested change
EPG_parse_table(ctx, ctx->epg_buffers[i].buffer, ctx->epg_buffers[i].buffer_length);
// Validate buffer before parsing to avoid out-of-bounds reads
if (ctx->epg_buffers[i].buffer_length > 0)
{
unsigned char pointer_field = (unsigned char)ctx->epg_buffers[i].buffer[0];
if ((size_t)pointer_field + 1 < (size_t)ctx->epg_buffers[i].buffer_length)
{
EPG_parse_table(ctx,
ctx->epg_buffers[i].buffer,
ctx->epg_buffers[i].buffer_length);
}
}

Copilot uses AI. Check for mistakes.
Validate buffer_length > 0 and pointer_field+1 < buffer_length before
calling EPG_parse_table() to avoid out-of-bounds reads on truncated or
malformed EIT sections at stream end.

Addresses Copilot review comment on CCExtractor#2166.
@ccextractor-bot
Copy link
Collaborator

CCExtractor CI platform finished running the test files on linux. Below is a summary of the test results, when compared to test for commit f377be9...:
Report Name Tests Passed
Broken 13/13
CEA-708 14/14
DVB 7/7
DVD 3/3
DVR-MS 2/2
General 27/27
Hardsubx 1/1
Hauppage 3/3
MP4 3/3
NoCC 10/10
Options 86/86
Teletext 21/21
WTV 13/13
XDS 34/34

Congratulations: Merging this PR would fix the following tests:

  • ccextractor --autoprogram --out=srt --latin1 --quant 0 85271be4d2..., Last passed: Never
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla dab1c1bd65..., Last passed: Never
  • ccextractor --out=srt --latin1 --autoprogram 29e5ffd34b..., Last passed: Never
  • ccextractor --out=spupng c83f765c66..., Last passed: Never
  • ccextractor --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9..., Last passed: Never
  • ccextractor --startcreditsnotbefore 1 --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9..., Last passed: Never
  • ccextractor --startcreditsnotafter 2 --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9..., Last passed: Never
  • ccextractor --startcreditsforatleast 1 --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9..., Last passed: Never
  • ccextractor --startcreditsforatmost 2 --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9..., Last passed: Never

All tests passed completely.

Check the result page for more info.

@ccextractor-bot
Copy link
Collaborator

CCExtractor CI platform finished running the test files on windows. Below is a summary of the test results, when compared to test for commit f377be9...:
Report Name Tests Passed
Broken 13/13
CEA-708 14/14
DVB 6/7
DVD 3/3
DVR-MS 2/2
General 25/27
Hardsubx 1/1
Hauppage 3/3
MP4 3/3
NoCC 10/10
Options 80/86
Teletext 21/21
WTV 13/13
XDS 34/34

Your PR breaks these cases:

  • ccextractor --autoprogram --out=srt --latin1 --quant 0 85271be4d2...
  • ccextractor --autoprogram --out=ttxt --latin1 --ucla dab1c1bd65...
  • ccextractor --out=srt --latin1 --autoprogram 29e5ffd34b...
  • ccextractor --out=spupng c83f765c66...
  • ccextractor --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9...
  • ccextractor --startcreditsnotbefore 1 --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9...
  • ccextractor --startcreditsnotafter 2 --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9...
  • ccextractor --startcreditsforatleast 1 --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9...
  • ccextractor --startcreditsforatmost 2 --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9...

It seems that not all tests were passed completely. This is an indication that the output of some files is not as expected (but might be according to you).

Check the result page for more info.

@Varadraj75
Copy link
Contributor Author

The Windows CI failures are pre-existing and unrelated to this change:

  • All failing tests show Empty path name is not legal — a Windows path
    handling issue in the test runner
  • All failing tests show "Last passed: Never" in the CI bot summary,
    confirming they were already failing on master before this PR
  • Linux CI passes all tests cleanly

This PR only modifies EPG_free() in ts_tables_epg.c which has no
interaction with path handling or the failing test cases.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG]: Last EIT section never parsed — EPG_free() discards pending epg_buffers without flushing

3 participants