Vote #77103
完了Received text attachments doesn't hold the original encoding on Ruby >= 2.1
0%
説明
Hello,
if I send an email with .txt file encoded in ISO 8859-2, it's received from the mailbox and saved to /files, but the encoding is different (some characters are lost). I think this is originaly caused by the utf-8 header added by email clients. In my opinion attachments shouln't be reencoded. Am I missed somethig?
Attached patch fixes the issue.
Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: base64 Environment: Redmine version 3.2.0.devel.15058 Ruby version 2.1.5-p273 (2014-11-13) [i386-mingw32] Rails version 4.2.5 Environment production Database adapter Mysql2 SCM: Subversion 1.8.13 Git 1.9.5 Filesystem Redmine plugins: no plugin installed
journals
Could you add test?
* source:tags/3.2.0/test/unit/mail_handler_test.rb#L563
* source:tags/3.2.0/test/fixtures/mail_handler/gmail_with_attachment_iso-8859-1.eml
--------------------------------------------------------------------------------
Done
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
I cannot reproduce on trunk r15148 with ruby-1.9.3-p551.
Test passes *without* attachment:mail_handler.rb.patch .
You can see latin2 encoded file in files/ directory with following change.
<pre><code class="diff">
diff --git a/app/models/attachment.rb b/app/models/attachment.rb
--- a/app/models/attachment.rb
+++ b/app/models/attachment.rb
@@ -53,7 +53,7 @@ class Attachment < ActiveRecord::Base
@@thumbnails_storage_path = File.join(Rails.root, "tmp", "thumbnails")
before_create :files_to_final_location
- after_rollback :delete_from_disk, :on => :create
+ # after_rollback :delete_from_disk, :on => :create
after_commit :delete_from_disk, :on => :destroy
# Returns an unsaved copy of the attachment
</code></pre>
<pre>
$ ruby test/unit/mail_handler_test.rb -n test_mail_with_attachment_latin2
Run options: -n test_mail_with_attachment_latin2 --seed 14783
# Running:
.
Finished in 2.025752s, 0.4936 runs/s, 3.4555 assertions/s.
1 runs, 7 assertions, 0 failures, 0 errors, 0 skips
$ cat files/2016/02/160208173558_yourfile.txt | iconv -f latin2 -t utf8
příliš žluoučký kůn
</pre>
--------------------------------------------------------------------------------
retested against revision r15148 without mail_handler.rb.patch
MailHandlerTest#test_mail_with_attachment_latin2 [C:/redmine svn/test/unit/mail_handler_test.rb:590]:
Expected: 19
Actual: 12
About your application's environment
Rails version 4.2.5.1
Ruby version 2.1.5-p273 (x64-mingw32)
RubyGems version 2.2.3
Rack version 1.6.4
Middleware Rack::Sendfile, ActionDispatch::Static, Rack::Lock, #<
ActiveSupport::Cache::Strategy::LocalCache::Middleware:0x000000094378e0>, Rack::
Runtime, Rack::MethodOverride, ActionDispatch::RequestId, Rails::Rack::Logger, A
ctionDispatch::ShowExceptions, ActionDispatch::DebugExceptions, ActionDispatch::
RemoteIp, ActionDispatch::Reloader, ActionDispatch::Callbacks, ActiveRecord::Con
nectionAdapters::ConnectionManagement, ActiveRecord::QueryCache, ActionDispatch:
:Cookies, ActionDispatch::Session::CookieStore, ActionDispatch::Flash, ActionDis
patch::ParamsParser, ActionDispatch::XmlParamsParser, Rack::Head, Rack::Conditio
nalGet, Rack::ETag, Rack::ContentLength, RequestStore::Middleware, OpenIdAuthent
ication
Application root C:/redmine svn
Environment development
Database adapter mysql2
Database schema version 20151031095005
Environment:
Redmine version 3.2.0.devel
Ruby version 2.1.5-p273 (2014-11-13) [x64-mingw32]
Rails version 4.2.5.1
Environment production
Database adapter Mysql2
SCM:
Git 1.9.5
Filesystem
Redmine plugins:
no plugin installed
no other tests are failing (except IssueNestedSetConcurrencyTest which supposed to be a known problem on mysql) and with mail_handler.rb.patch applied this one passes as well.
IssueNestedSetConcurrencyTest#test_concurrency [C:/redmine svn/test/unit/issue_nested_set_concurrency_test.rb:45]:
Expected "Mysql2::Error: Deadlock found when trying to get lock; try restarting
transaction: SELECT `issues`.`id` FROM `issues` WHERE (root_id IN (SELECT root_i
d FROM issues WHERE id IN (800,794))) ORDER BY `issues`.`id` ASC FOR UPDATE" to
be nil.
I attached both attachments, the original and the one saved to files (encoded.txt). Maybe it depends on the environment, I'll try to find out.
--------------------------------------------------------------------------------
@Toshi MARUYAMA - could you retest it on a more recent Ruby version?
passed - ruby 1.9.3p551 (2014-11-13) [i386-mingw32]
failed - ruby 2.1.5p273 (2014-11-13 revision 48405) [i386-mingw32]
failed - ruby 2.2.3p173 (2015-08-18 revision 51636) [x64-mingw32]
failed - jruby 9.0.5.0-SNAPSHOT (2.2.3) 2016-01-20 775b79d Java HotSpot(TM) 64-Bit Server
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
Committed in trunk, thanks.
I have reported to upstream.
https://github.com/mikel/mail/issues/972
--------------------------------------------------------------------------------