Vote #78473
完了CSV Importer - handle UndefinedConversionErrors
0%
説明
The CSV import already handles a couple of Exceptions that may happen when the user-selected encoding does not match the real encoding of the CSV (or it contains otherwise illegal byte sequences). We recently encountered @Encoding::UndefinedConversionError@, which is not yet caught by that error handling. The attached patches add a test case and a fix for that.
journals
We can reproduce the problem with the test in attachment:0001-adds-failing-test-case-for-import-with-wrong-encodin.patch. Setting target version to 3.2.7.
By the way, the encoding of the file @invalid-Shift_JIS.csv@ is valid CP932. Shift_JIS and CP932 are almost identical, but CP932 includes more characters.
<pre>
$ iconv -f cp932 -t utf8 test/fixtures/files/invalid-Shift_JIS.csv
①
</pre>
--------------------------------------------------------------------------------
Go MAEDA wrote:
> We can reproduce the problem with the test in attachment:0001-adds-failing-test-case-for-import-with-wrong-encodin.patch. Setting target version to 3.2.7.
The test passes for me with ruby 2.0 (I would get a Encoding::InvalidByteSequenceError). Maybe we could simply rescue @EncodingError@ which is the base class for encoding errors?
--------------------------------------------------------------------------------
Jean-Philippe Lang wrote:
> Maybe we could simply rescue @EncodingError@ which is the base class for encoding errors?
I agree, caching EncodingError is better. We can simplify the code and catch all errors caused by iconv.
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------