Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • something weird with replace + subinstr

    Dear statuses,

    I'm currently cleaning some string data and for that purpose I need to recome the accents (á, é.... à, è.... â, ê....). I have always used the following command for this task:

    Code:
    foreach var of varlist school{
    
        replace `var' = subinstr(`var', "á", "a",.)
        replace `var' = subinstr(`var', "é", "e",.)
        replace `var' = subinstr(`var', "í", "i",.)
        replace `var' = subinstr(`var', "ó", "o",.)
        replace `var' = subinstr(`var', "ú", "u",.)
        
        replace `var' = subinstr(`var', "à", "a",.)
        replace `var' = subinstr(`var', "è", "e",.)
        replace `var' = subinstr(`var', "ì", "i",.)
        replace `var' = subinstr(`var', "ò", "o",.)
        replace `var' = subinstr(`var', "ù", "u",.)
    
        replace `var' = subinstr(`var', "â", "a",.)
        replace `var' = subinstr(`var', "ê", "e",.)
        replace `var' = subinstr(`var', "î", "i",.)
        replace `var' = subinstr(`var', "ô", "o",.)
        replace `var' = subinstr(`var', "û", "u",.)
        
        replace `var' = subinstr(`var', "ä", "a",.)
        replace `var' = subinstr(`var', "ë", "e",.)
        replace `var' = subinstr(`var', "ï", "i",.)
        replace `var' = subinstr(`var', "ö", "o",.)
        replace `var' = subinstr(`var', "ü", "u",.)
    }
    However, something weird is going on here given that some accents prevail while others are replaced by letters with no accents, as I wanted (see the examples below, in green the string names for which the command works, in red the ones that were not effected by the command, both before and after the command is applied).

    Original sample:

    Code:
    * Example generated by -dataex-. For more info, type help dataex
    clear
    input str37 school
    "Adama Diallo"                    
    "El Hadji Mamadou Ndiaye "        
    "Yoff-Village "                   
    "Ouakam 2 "                       
    "Abbé Arsène Fridoil "        
    "Abbé Pierre Sock "             
    "Abdoulaye Mathurin Diop "        
    "Blaise Diagne "                  
    "BST Point E "                    
    "Cheikh Awa Balla Mbacké "      
    "El Hadji Malick Sy "             
    "El Hadji Mansour Sy Malick"      
    "Lamine Guèye "                 
    "Mame Thierno Birahim Mbacké"    
    "Martin Luther King "             
    "Ousmane Diop Coumba Pathé"      
    "Ahmet Sy Malick "                
    "BST Liberté 3 "                
    "Manguiers "                      
    "Adama Ndiaye "                   
    "Alioune Diop "                   
    "Amadou Trawaré "               
    "Dalifort "                       
    "David Diop "                     
    "Docteur Samba Guèye "          
    "El Hadji Badara Mbaye Kaba "     
    "Hann "                           
    "HLM 4/C "                        
    "Liberté 6/C "                  
    "Ousmane Socé Diop "            
    "Banque Islamique "               
    "P.A. Unité 5 "                 
    "Cité Enseignants "             
    "El Hadji Ogo Diop"               
    "Joseph Félix Corréa/A "      
    "Joseph Félix Corréa/B "      
    "Ndiarka Diagne "                 
    "Pikine Est/A "                   
    "Pikine Est/B "                   
    "Serigne Cheikh Anta Mbacké"    
    "Keur Massar 1 "                  
    "Keur Massar 2 "                  
    "Keur Massar 3 "                  
    "Keur Massar Village "            
    "Keur Massar zone de Recasement"  
    "Malika "                         
    "Malika plage "                   
    "Momar Marème Diop "            
    "Yeumbeul 2 "                     
    "Keur Massar Unité 9 "          
    "Cambérène "                  
    "El Hadji Ibrahima thiaw "        
    "Grand-Yoff "                     
    "Hlm Grand Yoff "                 
    "Parcelles Assainies Unité 18 " 
    "Parcelles Assainies Unité 19 " 
    "Patte d'Oie Builders "           
    "Scat-Urbam"                      
    "Talibou Dabo "                   
    "Parcelles Assainies Unité 20 " 
    "Chérif Mouhamed El Habib Tijani"
    "Fadilou Diop"                    
    "Mame yelli Badiane"              
    "Ecole 9"                         
    "Abdoulaye Sadji"                 
    "Arafat 2"                        
    "BST Rufisque"                    
    "Camp Marchand Arafat"            
    "Cité Tacko"                     
    "Matar Seck"                      
    "Maurice Guèye"                  
    "Momar Sène Waly"                
    "Pionniers du Syndicalisme"       
    "Tafsir Niao Faye"                
    "Dare Salam"                      
    "Bambylor"                        
    "Bargny"                          
    "Déni Birame Ndao"               
    "Diam Niadio"                     
    "Keur Ndiaye Lô"                 
    "Kip barrage (Commune Bargny)"    
    "Kounoune"                        
    "Niaga"                           
    "Plan Jaxaay"                     
    "Sangalkam"                       
    "Sébi Gare (Commune Sébikotane)"
                    
    end

    After the command aforementioned (replace + subinstr)

    Code:
    * Example generated by -dataex-. For more info, type help dataex
    clear
    input str37 school
    "Adama Diallo"                   
    "El Hadji Mamadou Ndiaye "       
    "Yoff-Village "                  
    "Ouakam 2 "                      
    "Abbé Arsène Fridoil "       
    "Abbé Pierre Sock "            
    "Abdoulaye Mathurin Diop "       
    "Blaise Diagne "                 
    "BST Point E "                   
    "Cheikh Awa Balla Mbacké "     
    "El Hadji Malick Sy "            
    "El Hadji Mansour Sy Malick"     
    "Lamine Guèye "                
    "Mame Thierno Birahim Mbacke" 
    "Martin Luther King "            
    "Ousmane Diop Coumba Pathe"      
    "Ahmet Sy Malick "               
    "BST Liberté 3 "               
    "Manguiers "                     
    "Adama Ndiaye "                  
    "Alioune Diop "                  
    "Amadou Trawaré "              
    "Dalifort "                      
    "David Diop "                    
    "Docteur Samba Guèye "         
    "El Hadji Badara Mbaye Kaba "    
    "Hann "                          
    "HLM 4/C "                       
    "Liberté 6/C "                 
    "Ousmane Socé Diop "           
    "Banque Islamique "              
    "P.A. Unité 5 "                
    "Cité Enseignants "            
    "El Hadji Ogo Diop"              
    "Joseph Félix Corréa/A "     
    "Joseph Félix Corréa/B "     
    "Ndiarka Diagne "                
    "Pikine Est/A "                  
    "Pikine Est/B "                  
    "Serigne Cheikh Anta Mbacké"   
    "Keur Massar 1 "                 
    "Keur Massar 2 "                 
    "Keur Massar 3 "                 
    "Keur Massar Village "           
    "Keur Massar zone de Recasement" 
    "Malika "                        
    "Malika plage "                  
    "Momar Marème Diop "           
    "Yeumbeul 2 "                    
    "Keur Massar Unité 9 "         
    "Cambérène "                 
    "El Hadji Ibrahima thiaw "       
    "Grand-Yoff "                    
    "Hlm Grand Yoff "                
    "Parcelles Assainies Unité 18 "
    "Parcelles Assainies Unité 19 "
    "Patte d'Oie Builders "          
    "Scat-Urbam"                     
    "Talibou Dabo "                  
    "Parcelles Assainies Unité 20 "
    "Cherif Mouhamed El Habib Tijani"
    "Fadilou Diop"                   
    "Mame yelli Badiane"             
    "Ecole 9"                        
    "Abdoulaye Sadji"                
    "Arafat 2"                       
    "BST Rufisque"                   
    "Camp Marchand Arafat"           
    "Cite Tacko"                     
    "Matar Seck"                     
    "Maurice Gueye"                  
    "Momar Sene Waly"                
    "Pionniers du Syndicalisme"      
    "Tafsir Niao Faye"               
    "Dare Salam"                     
    "Bambylor"                       
    "Bargny"                         
    "Deni Birame Ndao"               
    "Diam Niadio"                    
    "Keur Ndiaye Lo"                 
    "Kip barrage (Commune Bargny)"   
    "Kounoune"                       
    "Niaga"                          
    "Plan Jaxaay"                    
    "Sangalkam"                      
    "Sebi Gare (Commune Sebikotane)" 
    "Tivaouane Peulh"                
    "Yene"                           
    "Ponty"                          
    "Gorom 1"                        
    "Niacourab"                      
    "Dougar"                         
    "CAPEC"                          
    "Diamaguene"                     
    "Martyrs C"                      
    "Mbao"                           
    "Mbao Kamb"                      
    "Ndiawar Diagne"                 
    "Thiaroye 2"                     
    "Thiaroye 44"                    
    end
    I have tried many different ways to fix this buy anything works. You may thing that the space just after the "é" may cause this but even after deleting this space or using the command replace + subinstr for "é " accounting for the space, it doesn't work, besides that, that wouldn't explain why "Joseph Félix Corréa/A " "Joseph Félix Corréa/B " remain unchanged.

    Every suggestion to solve this mistery is more than welcome

    Thank you in advance,

    Daniel.

  • #2
    Try

    Code:
    usubinstr()
    If that does not work, you need more advanced Unicode functions, e.g., ustrnormalize()

    This is an extensive topic; read

    Code:
    help unicode
    for the full story.

    Comment


    • #3
      Code:
      gen wanted= ustrto(ustrnormalize(school, "nfd"), "ascii", 2)
      Res.:

      Code:
      . l, sep(0)
      
           +-------------------------------------------------------------------+
           |                          school                            wanted |
           |-------------------------------------------------------------------|
        1. |                    Adama Diallo                      Adama Diallo |
        2. |        El Hadji Mamadou Ndiaye           El Hadji Mamadou Ndiaye  |
        3. |                   Yoff-Village                      Yoff-Village  |
        4. |                       Ouakam 2                          Ouakam 2  |
        5. |            Abbé Arsène Fridoil               Abbe Arsene Fridoil  |
        6. |               Abbé Pierre Sock                  Abbe Pierre Sock  |
        7. |        Abdoulaye Mathurin Diop           Abdoulaye Mathurin Diop  |
        8. |                  Blaise Diagne                     Blaise Diagne  |
        9. |                    BST Point E                       BST Point E  |
       10. |        Cheikh Awa Balla Mbacké           Cheikh Awa Balla Mbacke  |
       11. |             El Hadji Malick Sy                El Hadji Malick Sy  |
       12. |      El Hadji Mansour Sy Malick        El Hadji Mansour Sy Malick |
       13. |                   Lamine Guèye                      Lamine Gueye  |
       14. |     Mame Thierno Birahim Mbacké       Mame Thierno Birahim Mbacke |
       15. |             Martin Luther King                Martin Luther King  |
       16. |       Ousmane Diop Coumba Pathé         Ousmane Diop Coumba Pathe |
       17. |                Ahmet Sy Malick                   Ahmet Sy Malick  |
       18. |                  BST Liberté 3                     BST Liberte 3  |
       19. |                      Manguiers                         Manguiers  |
       20. |                   Adama Ndiaye                      Adama Ndiaye  |
       21. |                   Alioune Diop                      Alioune Diop  |
       22. |                 Amadou Trawaré                    Amadou Traware  |
       23. |                       Dalifort                          Dalifort  |
       24. |                     David Diop                        David Diop  |
       25. |            Docteur Samba Guèye               Docteur Samba Gueye  |
       26. |     El Hadji Badara Mbaye Kaba        El Hadji Badara Mbaye Kaba  |
       27. |                           Hann                              Hann  |
       28. |                        HLM 4/C                           HLM 4/C  |
       29. |                    Liberté 6/C                       Liberte 6/C  |
       30. |              Ousmane Socé Diop                 Ousmane Soce Diop  |
       31. |               Banque Islamique                  Banque Islamique  |
       32. |                   P.A. Unité 5                      P.A. Unite 5  |
       33. |               Cité Enseignants                  Cite Enseignants  |
       34. |               El Hadji Ogo Diop                 El Hadji Ogo Diop |
       35. |          Joseph Félix Corréa/A             Joseph Felix Correa/A  |
       36. |          Joseph Félix Corréa/B             Joseph Felix Correa/B  |
       37. |                 Ndiarka Diagne                    Ndiarka Diagne  |
       38. |                   Pikine Est/A                      Pikine Est/A  |
       39. |                   Pikine Est/B                      Pikine Est/B  |
       40. |      Serigne Cheikh Anta Mbacké        Serigne Cheikh Anta Mbacke |
       41. |                  Keur Massar 1                     Keur Massar 1  |
       42. |                  Keur Massar 2                     Keur Massar 2  |
       43. |                  Keur Massar 3                     Keur Massar 3  |
       44. |            Keur Massar Village               Keur Massar Village  |
       45. |  Keur Massar zone de Recasement    Keur Massar zone de Recasement |
       46. |                         Malika                            Malika  |
       47. |                   Malika plage                      Malika plage  |
       48. |              Momar Marème Diop                 Momar Mareme Diop  |
       49. |                     Yeumbeul 2                        Yeumbeul 2  |
       50. |            Keur Massar Unité 9               Keur Massar Unite 9  |
       51. |                      Cambérène                         Camberene  |
       52. |        El Hadji Ibrahima thiaw           El Hadji Ibrahima thiaw  |
       53. |                     Grand-Yoff                        Grand-Yoff  |
       54. |                 Hlm Grand Yoff                    Hlm Grand Yoff  |
       55. |   Parcelles Assainies Unité 18      Parcelles Assainies Unite 18  |
       56. |   Parcelles Assainies Unité 19      Parcelles Assainies Unite 19  |
       57. |           Patte d'Oie Builders              Patte d'Oie Builders  |
       58. |                      Scat-Urbam                        Scat-Urbam |
       59. |                   Talibou Dabo                      Talibou Dabo  |
       60. |   Parcelles Assainies Unité 20      Parcelles Assainies Unite 20  |
       61. | Chérif Mouhamed El Habib Tijani   Cherif Mouhamed El Habib Tijani |
       62. |                    Fadilou Diop                      Fadilou Diop |
       63. |              Mame yelli Badiane                Mame yelli Badiane |
       64. |                         Ecole 9                           Ecole 9 |
       65. |                 Abdoulaye Sadji                   Abdoulaye Sadji |
       66. |                        Arafat 2                          Arafat 2 |
       67. |                    BST Rufisque                      BST Rufisque |
       68. |            Camp Marchand Arafat              Camp Marchand Arafat |
       69. |                      Cité Tacko                        Cite Tacko |
       70. |                      Matar Seck                        Matar Seck |
       71. |                   Maurice Guèye                     Maurice Gueye |
       72. |                 Momar Sène Waly                   Momar Sene Waly |
       73. |       Pionniers du Syndicalisme         Pionniers du Syndicalisme |
       74. |                Tafsir Niao Faye                  Tafsir Niao Faye |
       75. |                      Dare Salam                        Dare Salam |
       76. |                        Bambylor                          Bambylor |
       77. |                          Bargny                            Bargny |
       78. |                Déni Birame Ndao                  Deni Birame Ndao |
       79. |                     Diam Niadio                       Diam Niadio |
       80. |                  Keur Ndiaye Lô                    Keur Ndiaye Lo |
       81. |    Kip barrage (Commune Bargny)      Kip barrage (Commune Bargny) |
       82. |                        Kounoune                          Kounoune |
       83. |                           Niaga                             Niaga |
       84. |                     Plan Jaxaay                       Plan Jaxaay |
       85. |                       Sangalkam                         Sangalkam |
       86. |  Sébi Gare (Commune Sébikotane)    Sebi Gare (Commune Sebikotane) |
           +-------------------------------------------------------------------+

      Comment


      • #4
        Andrew Musau daniel klein .Thanks to both, I could solve it identifying the problem using unicode. I'have just realised I forgot to respond to both, my apologies.

        Daniel.

        Comment

        Working...
        X