Opened 10 years ago

Closed 10 years ago

#2305 closed enhancement (fixed)

Issues with importing messages from Translatewiki

Reported by: avar Owned by: Tom Hughes
Priority: major Milestone:
Component: website Version:
Keywords: Cc:

Description

I'm filing a bug because I need somewhere to host these huge patches:

How to sync from Translatewiki manually:

for i in $(ls *yml | grep -v en.yml); do echo wget "http://translatewiki.net/w/i.php?title=Special%3ATranslate&task=export-to-file&group=out-osm&language=$(echo $i | sed s/.yml// | tr 'A-Z' 'a-z')" -O $i; done

Attached is a unified diff patch and a pretty-formatted one to show what's changed.

Attachments (2)

all.patch.gz (103.7 KB) - added by avar 10 years ago.
Current export from Translatewiki
first-translatewiki-patch.patch.gz (101.5 KB) - added by avar 10 years ago.
pseudo-patch I'm about to apply to trunk, for review

Download all attachments as: .zip

Change History (7)

comment:1 Changed 10 years ago by avar

Actually trac doesn't take attachments of this size, here they are:

Some URLs for example are being removed. This is a matter of policy in the exporter. I've already fixed some of these (e.g. not nuking wiki URLs).

comment:2 Changed 10 years ago by avar

Summary: [PATCH] Import messages from TranslatewikiIssues with importing messages from Translatewiki

Here are some outstanding bugs with the patch & comments:

Messages nuked because Translatewiki doesn't understand rails plural forms:

    -browse.changeset_details.has_nodes: "Té els següents {{count}} nodes:"                                                                                  
    -browse.changeset_details.has_relations: "Té les següents {{count}} relacions:"                                                                          
    -browse.changeset_details.has_ways: "Té els següents {{count}} camins:"

Leaking "FUZZY" into messages:

    -activerecord.attributes.user.display_name: Name anzeigen                                                                                                
    +activerecord.attributes.user.display_name: "!!Fuzzy!!Name anzeigen"

This is OK, we should be nuking whitespace:

    -browse.node_details.coordinates: "Συντεταγμένες: "                                                                                                      
    +browse.node_details.coordinates: "Συντεταγμένες:"

We should nuke stuff equivalent to en.yml:

     export.start_rjs.export: Εξαγωγή                                                                                                                        
    -html.dir: ltr                                                                                                                                           
     map.coordinates: "Συντεταγμένες:"

    -layouts.project_name.h1: OpenStreetMap                                                                                                                  
    -layouts.project_name.title: OpenStreetMap

OK, nuking no longer used messages (not in en.yml):

    -changeset.list_bbox.history: Historial                                                                                                                  
    -changeset.list_bbox.no_area_specified: No se especificó un Área                                                                                         
    -changeset.list_bbox.no_changesets: Sin conjuntos de cambios                                                                                             
    -changeset.list_bbox.recent_changes: Cambios Recientes                                                                                                   
    -changeset.list_bbox.show_area_box: mostrar caja                                                                                                         
    -changeset.list_bbox.view_tab: pestaña vista                                                                                                             
    -changeset.list_bbox.view_the_map: ver el mapa                                                                                                           
    -changeset.list_user.edits_by_username: Ediciones hechas por {{username_link}}                                                                           
    -changeset.list_user.for_all_changes: Para los cambios de todos los usuarios vea {{recent_changes_link}}                                                 
    -changeset.list_user.no_visible_edits_by: "{{name}} no ha hecho ediciones visibles."                                                                     
    -changeset.list_user.recent_changes: Cambios Recientes 

Otherwise it seems fine.

comment:3 Changed 10 years ago by avar

Here's how I review these:

diff old:

for i in $(ls *yml | grep -v en.yml); do perl ../../script/locale/diff --dump-flat $i > $i.0 ;done

get new:

for i in $(ls *yml | grep -v en.yml); do wget "http://translatewiki.net/w/i.php?title=Special%3ATranslate&task=export-to-file&group=out-osm&language=$(echo $i | sed s/.yml// | tr 'A-Z' 'a-z')" -O $i; done

diff new:

for i in $(ls *yml | grep -v en.yml); do perl ../../script/locale/diff --dump-flat $i > $i.1 ;done

get a patch to review:

for i in $(ls *yml | grep -v en.yml); do diff -ru $i.*; done

Changed 10 years ago by avar

Attachment: all.patch.gz added

Current export from Translatewiki

comment:4 in reply to:  2 Changed 10 years ago by avar

Replying to avarab@gmail.com:

Update: These are now the existing bugs:

    -browse.changeset_details.has_nodes: "Té els següents {{count}} nodes:"                                                                                  
    -browse.changeset_details.has_relations: "Té les següents {{count}} relacions:"                                                                          
    -browse.changeset_details.has_ways: "Té els següents {{count}} camins:"

This is still the case.

Leaking "FUZZY" into messages:

This was an issue of a user manually inserting "FUZZY" into a message, not the export being broken.

Here are some more issues:

It's removing messages which were on the OpenStreetMap.yml ignore list during import, e.g.:

-layouts.help_wiki_url: http://wiki.openstreetmap.org/wiki/RU:Main_Page

and:

-layouts.intro_3_bytemark: bytemark                                                                                                          
-layouts.intro_3_ucl: Цэнтр UCL VR

It doesn't grok that site.key.table.entry.* can be an array or a string:

Original in en.yml:

school:
  - School
  - university

Version in zh-TW.yml:

school: "學校;大學"

It removes this because the key doesn't exist, from zh-TW.yml diff:

-site.key.table.entry.school: 學校;大學

comment:5 Changed 10 years ago by avar

Resolution: fixed
Status: newclosed

Outstanding merge issues were solved with a script in [17828] and subsequent commits. Closing this.

Changed 10 years ago by avar

pseudo-patch I'm about to apply to trunk, for review

Note: See TracTickets for help on using tickets.