tickets #57530

Redmine cannot handle UTF8 characters in the lastname

Added by dancermak 5 months ago. Updated about 1 month ago.

Status:WorkableStart date:30/09/2019
Priority:NormalDue date:
Assignee:tuanpembual% Done:

10%

Category:redmine
Target version:-
Duration:

Description

I can only login to progress.opensuse.org when I convert my lastname from "Čermák" to "Cermak", otherwise redmine dies with (https://paste.opensuse.org/b4d3c7a7):

Started GET "/login" for 89.12.147.145 at Mon Sep 30 08:52:01 +0000 2019
Processing by AccountController#login as HTML
Current user: anonymous
Completed 500 Internal Server Error in 336.5ms

ActiveRecord::StatementInvalid (Mysql2::Error: Incorrect string value: '\xC4\x8Cerm\xC3...' for column 'lastname' at row 1: UPDATE users SET login = 'dancermak', updated_on = '2019-09-30 08:52:01', lastname = 'Čermák'
WHERE users.type IN ('User', 'AnonymousUser') AND users.id = 33662):
activerecord (3.2.17) lib/active_record/connection_adapters/abstract_mysql_adapter.rb:245:in query'
activerecord (3.2.17) lib/active_record/connection_adapters/abstract_mysql_adapter.rb:245:in
execute'
activerecord (3.2.17) lib/active_record/connection_adapters/abstract_adapter.rb:280:in log'
activesupport (3.2.17) lib/active_support/notifications/instrumenter.rb:20:in
instrument'
activerecord (3.2.17) lib/active_record/connection_adapters/abstract_adapter.rb:275:in log'
activerecord (3.2.17) lib/active_record/connection_adapters/abstract_mysql_adapter.rb:245:in
execute'
activerecord (3.2.17) lib/active_record/connection_adapters/mysql2_adapter.rb:213:in execute'
activerecord (3.2.17) lib/active_record/connection_adapters/mysql2_adapter.rb:240:in
exec_update'
activerecord (3.2.17) lib/active_record/connection_adapters/abstract/database_statements.rb:96:in update'
activerecord (3.2.17) lib/active_record/connection_adapters/abstract/query_cache.rb:14:in
update'
activerecord (3.2.17) lib/active_record/persistence.rb:359:in update'
activerecord (3.2.17) lib/active_record/locking/optimistic.rb:68:in
update'
activerecord (3.2.17) lib/active_record/attribute_methods/dirty.rb:74:in update'
activerecord (3.2.17) lib/active_record/timestamp.rb:71:in
update'
activerecord (3.2.17) lib/active_record/callbacks.rb:272:in update'
activesupport (3.2.17) lib/active_support/callbacks.rb:403:in
_run__564565134__update__4__callbacks'
activesupport (3.2.17) lib/active_support/callbacks.rb:405:in send'
activesupport (3.2.17) lib/active_support/callbacks.rb:405:in
__run_callback'
activesupport (3.2.17) lib/active_support/callbacks.rb:385:in `_run_update_callbacks'

History

#1 Updated by okurz 5 months ago

  • Private changed from Yes to No

#2 Updated by okurz 5 months ago

Looks like a codeset or encoding problem.

The general mysql encoding settings should be fine:

mysql> show variables like 'char%';
+--------------------------+------------------------------+
| Variable_name            | Value                        |
+--------------------------+------------------------------+
| character_set_client     | utf8                         |
| character_set_connection | utf8                         |
| character_set_database   | utf8mb4                      |
| character_set_filesystem | binary                       |
| character_set_results    | utf8                         |
| character_set_server     | utf8mb4                      |
| character_set_system     | utf8                         |
| character_sets_dir       | /usr/share/mariadb/charsets/ |
+--------------------------+------------------------------+
8 rows in set (0.01 sec)

however the one from the progress database might be the problem:

mysql> show variables like 'char%';
+--------------------------+------------------------------+
| Variable_name            | Value                        |
+--------------------------+------------------------------+
| character_set_client     | utf8                         |
| character_set_connection | utf8                         |
| character_set_database   | utf8                         |
| character_set_filesystem | binary                       |
| character_set_results    | utf8                         |
| character_set_server     | utf8mb4                      |
| character_set_system     | utf8                         |
| character_sets_dir       | /usr/share/mariadb/charsets/ |
+--------------------------+------------------------------+
8 rows in set (0.01 sec)

so character_set_database is not set to utf8mb4. https://stackoverflow.com/a/34854453 might be the answer or https://www.redmine.org/issues/10772#note-7 . Potentially also the database is too old itself? See https://www.redmine.org/issues/21398#note-12

so what are the opinions of the other heroes?

#3 Updated by dancermak 3 months ago

Any updates on this?

#4 Updated by lrupp 3 months ago

  • Status changed from New to Workable
  • Assignee set to tuanpembual

I think we should check, if the new system has this problem as well...

Assigning to our current Redmine hero!

#5 Updated by tuanpembual 3 months ago

Thank you. I will try reproduce this scenario.

#6 Updated by tuanpembual 3 months ago

Hi,

I have checked on new progress (redmine 3.4.12.stable).
This issue still happen. I try to change my last name in my account to use "Čermák" and Redmine show 500 error code.

I do some alter sql query.

ALTER DATABASE redmine_default CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
ALTER TABLE users CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;

add set encode on database.yml.
production:
encoding: utf8mb4

Restart redmine.
And it is working now.

Estu~

#7 Updated by dancermak 3 months ago

Unfortunately it is still broken for me. I just changed my last name to
'Čermák' and now I cannot login to progress anymore. Changing the name
back to 'Cermak' doesn't solve the issue.

#8 Updated by tuanpembual 3 months ago

Hi Dancermak,

Sorry, we have missing understanding.
New progress redmine not yet on production use, still using dummy url. and I only fixed on new progress. So you still get error 500.

About new progress redmine, I hope it can release as soon as possible.

#9 Updated by dancermak 3 months ago

admin@opensuse.org writes:

[openSUSE Tracker]

Issue #57530 has been updated by tuanpembual.


Hi Dancermak,


Sorry, we have missing understanding.

New progress redmine not yet on production use, still using dummy url. and I only fixed on new progress. So you still get error 500.

No need to apologize, I should have read your message properly ;-).

About new progress redmine, I hope it can release as soon as possible.

Looking forward to that! But no need to hurry, this is really just a
minor inconvenience for me.

#10 Updated by lrupp about 1 month ago

  • % Done changed from 0 to 10

JFYI: I upgraded/converted the database already. As utf8 is a subset of utf8mb4, this should not harm. But the changes in Redmine's productive config need a newer Ruby mysql client - otherwise the server fails with: "Character set 'utf8mb4' is not a compiled character set and is not specified in the '/usr/share/mysql/charsets/Index.xml' file".

So: the sooner we migrate to the new Redmine instance, the better :-)

Also available in: Atom PDF