Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove BOM in CsvFileIterator #8

Open
BenMorel opened this issue Oct 23, 2019 · 1 comment
Open

Remove BOM in CsvFileIterator #8

BenMorel opened this issue Oct 23, 2019 · 1 comment

Comments

@BenMorel
Copy link
Member

@BenMorel BenMorel commented Oct 23, 2019

When reading a file with an UTF-8 BOM (or any other BOM, for that matter) , the BOM is part of the fgetcsv() read, which corrupts the header row:

  • the BOM is part of the first column's name
  • if an enclosure character (e.g. ") is used, it's not detected for the first column name

Note sure whether removing the BOM should be part of this library, or whether we should rely on a filter.

@BenMorel
Copy link
Member Author

@BenMorel BenMorel commented Oct 23, 2019

Sample UTF-8 BOM filter:

class BOMFilter extends \php_user_filter
{
    private $checked = false;

    public function onCreate(): bool
    {
        return true;
    }

    public function onClose(): void
    {
    }

    public function filter($in, $out, &$consumed, $closing): int
    {
        while ($bucket = stream_bucket_make_writeable($in)) {
            if (! $this->checked) {
                if (substr($bucket->data, 0, 3) === "\xEF\xBB\xBF") {
                    $bucket->data = substr($bucket->data, 3);
                }

                $this->checked = true;
            }

            $consumed += $bucket->datalen;
            stream_bucket_append($out, $bucket);
        }

        return \PSFS_PASS_ON;
    }
}

Usage:

stream_filter_register('bom-filter', BOMFilter::class);

$fp = fopen($file, 'rb');
stream_filter_append($fp, 'bom-filter');
$rows = new CsvFileIterator($fp);
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
1 participant
You can’t perform that action at this time.