Project

General

Profile

Actions
Copy link

tickets #57530

closed

Redmine cannot handle UTF8 characters in the lastname

Added by dancermak over 5 years ago. Updated almost 5 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
tuanpembual
Category:
Redmine
Target version:
-
Start date:
2019-09-30
Due date:
% Done:

10%

Estimated time:

Description

I can only login to progress.opensuse.org when I convert my lastname from "Čermák" to "Cermak", otherwise redmine dies with (https://paste.opensuse.org/b4d3c7a7):

Started GET "/login" for 89.12.147.145 at Mon Sep 30 08:52:01 +0000 2019
Processing by AccountController#login as HTML
Current user: anonymous
Completed 500 Internal Server Error in 336.5ms

ActiveRecord::StatementInvalid (Mysql2::Error: Incorrect string value: '\xC4\x8Cerm\xC3...' for column 'lastname' at row 1: UPDATE users SET login = 'dancermak', updated_on = '2019-09-30 08:52:01', lastname = 'Čermák'
WHERE users.type IN ('User', 'AnonymousUser') AND users.id = 33662):
activerecord (3.2.17) lib/active_record/connection_adapters/abstract_mysql_adapter.rb:245:in query'
activerecord (3.2.17) lib/active_record/connection_adapters/abstract_mysql_adapter.rb:245:inexecute'
activerecord (3.2.17) lib/active_record/connection_adapters/abstract_adapter.rb:280:in log'
activesupport (3.2.17) lib/active_support/notifications/instrumenter.rb:20:ininstrument'
activerecord (3.2.17) lib/active_record/connection_adapters/abstract_adapter.rb:275:in log'
activerecord (3.2.17) lib/active_record/connection_adapters/abstract_mysql_adapter.rb:245:inexecute'
activerecord (3.2.17) lib/active_record/connection_adapters/mysql2_adapter.rb:213:in execute'
activerecord (3.2.17) lib/active_record/connection_adapters/mysql2_adapter.rb:240:inexec_update'
activerecord (3.2.17) lib/active_record/connection_adapters/abstract/database_statements.rb:96:in update'
activerecord (3.2.17) lib/active_record/connection_adapters/abstract/query_cache.rb:14:inupdate'
activerecord (3.2.17) lib/active_record/persistence.rb:359:in update'
activerecord (3.2.17) lib/active_record/locking/optimistic.rb:68:inupdate'
activerecord (3.2.17) lib/active_record/attribute_methods/dirty.rb:74:in update'
activerecord (3.2.17) lib/active_record/timestamp.rb:71:inupdate'
activerecord (3.2.17) lib/active_record/callbacks.rb:272:in update'
activesupport (3.2.17) lib/active_support/callbacks.rb:403:inrun564565134update4callbacks'
activesupport (3.2.17) lib/active_support/callbacks.rb:405:in send'
activesupport (3.2.17) lib/active_support/callbacks.rb:405:in_run_callback'
activesupport (3.2.17) lib/active_support/callbacks.rb:385:in `_run_update_callbacks'

Actions
Copy link
 #1

Updated by okurz over 5 years ago

  • Private changed from Yes to No
Actions
Copy link
 #2

Updated by okurz over 5 years ago

Looks like a codeset or encoding problem.

The general mysql encoding settings should be fine:

mysql> show variables like 'char%';
+--------------------------+------------------------------+
| Variable_name            | Value                        |
+--------------------------+------------------------------+
| character_set_client     | utf8                         |
| character_set_connection | utf8                         |
| character_set_database   | utf8mb4                      |
| character_set_filesystem | binary                       |
| character_set_results    | utf8                         |
| character_set_server     | utf8mb4                      |
| character_set_system     | utf8                         |
| character_sets_dir       | /usr/share/mariadb/charsets/ |
+--------------------------+------------------------------+
8 rows in set (0.01 sec)

however the one from the progress database might be the problem:

mysql> show variables like 'char%';
+--------------------------+------------------------------+
| Variable_name            | Value                        |
+--------------------------+------------------------------+
| character_set_client     | utf8                         |
| character_set_connection | utf8                         |
| character_set_database   | utf8                         |
| character_set_filesystem | binary                       |
| character_set_results    | utf8                         |
| character_set_server     | utf8mb4                      |
| character_set_system     | utf8                         |
| character_sets_dir       | /usr/share/mariadb/charsets/ |
+--------------------------+------------------------------+
8 rows in set (0.01 sec)

so character_set_database is not set to utf8mb4. https://stackoverflow.com/a/34854453 might be the answer or https://www.redmine.org/issues/10772#note-7 . Potentially also the database is too old itself? See https://www.redmine.org/issues/21398#note-12

so what are the opinions of the other heroes?

Actions
Copy link
 #3

Updated by dancermak about 5 years ago

Any updates on this?

Actions
Copy link
 #4

Updated by lrupp about 5 years ago

  • Status changed from New to Workable
  • Assignee set to tuanpembual

I think we should check, if the new system has this problem as well...

Assigning to our current Redmine hero!

Actions
Copy link
 #5

Updated by tuanpembual about 5 years ago

Thank you. I will try reproduce this scenario.

Actions
Copy link
 #6

Updated by tuanpembual about 5 years ago

Hi,

I have checked on new progress (redmine 3.4.12.stable).
This issue still happen. I try to change my last name in my account to use "Čermák" and Redmine show 500 error code.

I do some alter sql query.

ALTER DATABASE redmine_default CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE users CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

add set encode on database.yml.
production:
encoding: utf8mb4

Restart redmine.
And it is working now.

Estu~

Actions
Copy link
 #7

Updated by dancermak about 5 years ago

Unfortunately it is still broken for me. I just changed my last name to
'Čermák' and now I cannot login to progress anymore. Changing the name
back to 'Cermak' doesn't solve the issue.

Actions
Copy link
 #8

Updated by tuanpembual about 5 years ago

Hi Dancermak,

Sorry, we have missing understanding.
New progress redmine not yet on production use, still using dummy url. and I only fixed on new progress. So you still get error 500.

About new progress redmine, I hope it can release as soon as possible.

Actions
Copy link
 #9

Updated by dancermak about 5 years ago

admin@opensuse.org writes:

[openSUSE Tracker]
Issue #57530 has been updated by tuanpembual.

Hi Dancermak,

Sorry, we have missing understanding.
New progress redmine not yet on production use, still using dummy url. and I only fixed on new progress. So you still get error 500.

No need to apologize, I should have read your message properly ;-).

About new progress redmine, I hope it can release as soon as possible.

Looking forward to that! But no need to hurry, this is really just a
minor inconvenience for me.

Actions
Copy link
 #10

Updated by lrupp about 5 years ago

  • % Done changed from 0 to 10

JFYI: I upgraded/converted the database already. As utf8 is a subset of utf8mb4, this should not harm. But the changes in Redmine's productive config need a newer Ruby mysql client - otherwise the server fails with: "Character set 'utf8mb4' is not a compiled character set and is not specified in the '/usr/share/mysql/charsets/Index.xml' file".

So: the sooner we migrate to the new Redmine instance, the better :-)

Actions
Copy link
 #11

Updated by tuanpembual almost 5 years ago

Hi,

We done migrate to new redmine. Can you re-check this issue again?
Thank you

Actions
Copy link
 #12

Updated by dancermak almost 5 years ago

tuanpembual wrote:

Hi,

We done migrate to new redmine. Can you re-check this issue again?
Thank you

It appears to be working correctly, thank you!

Actions
Copy link
 #13

Updated by tuanpembual almost 5 years ago

  • Status changed from Workable to Resolved
Actions
Copy link

Also available in: Atom PDF