how to use regexm to analyse string variables

[Gliby Anna]

Dear All

Can anyone help me in finding an easier way to categories open ended questions.


an example of my data set is given below.

* Example generated by -dataex-. To install: ssc install dataex
clear
input str500 other
"not getting material for poultry farm"                                                                                                                                                               
"not getting available medical for poultry farm"                                                                                                                                                      
"need improvememt in the road infrastructure"                                                                                                                                                         
"road infrastructure improvement needed"                                                                                                                                                              
"the local government needs to recognise support us in having created employment in the community but instead ignore our herbal tea farm as a private entity"                                         
"needed road infrastructure improvement"                                                                                                                                                              
"needed road infrastructure improvement"                                                                                                                                                              
"road infrastructure improvement"                                                                                                                                                                     
"need of a veterinary surgeon"                                                                                                                                                                        
"government needs to paid half payment for private employers government also allowed to import third countries like expertise in traditional handicrafts get experience for local people "            
"not getting pass for cyprus"                                                                                                                                                                         
"govt restricted on bar licence hairing to others"                                                                                                                                                    
"not getting loan for rural areas macro trade business provide low interest that will benefits for their business"                                                                                    
"road infrastructure improvement"                                                                                                                                                                     
"location problem and facing problem to pay the interest for the business"                                                                                                                            
"poor druk air services"                                                                                                                                                                              
"if the business items sales in same rate then it would be benifites to the shopkeeper"                                                                                                               
"labor problems"                                                                                                                                                                                      
"non bhutanese restricted for sawmill workers if the government issue the labor permits then it would solve our labor problem"                                                                        
"if government strict to bar licence those given on hair its effect on hairing owner business"                                                                                                        
"problem of space for business purposes"                                                                                                                                                              
"tax rates is qute not equal and itz unfair"                                                                                                                                                          
"government to allow few nonbhutanese for sawmill works to get experience for bhutanese workers if government grant on half payment for business setup"                                               
"no safety measures for workers"                                                                                                                                                                      
"renewal rate for snooker is quite high compare to other business like general shop"                                                                                                                  
"business loan if government issue than it be benefits"                                                                                                                                               
"restrictions during tenders"                                                                                                                                                                         
"immigration problem"                                                                                                                                                                                 
"chefs needed to import from other countries to get experience for bhutanese workers non bhutanese needed for hotel maintenance and repairs indian workers to import from any gate immigration allows"
"vegetables sellers need safety place to store room for perishable items mostly in winter season"                                                                                                     
end

[Andrew Musau]

First, you need to know what you are looking for. The fact that you have tools that make search easier does not help you if you do not know what you are searching for. Presumably, these are survey responses to an open-ended question, so the question provides some context. You need to skim through the responses to get some common themes, then define keywords that capture these themes. For example, infrastructure related could be road, car, railway, railroad, train, air, aircraft airport, etc. Another theme could be business related, and the keywords here might be business, company, permit, license, tax, grant, tender, etc. Once you have identified common themes and keywords, you can create indicators for these.

gen infrastructure = regexm(" " + lower(other) + " ", "['!?,\. ](road|car|railway|railroad|train|air|aircraft|airport)['!?,\. ]")
gen business = regexm(" " + lower(other) + " ", "['!?,\. ](business|company|permit|license|tax|grant|tender)['!?,\. ]")

Some responses may be classified into two or more categories, and you have to sort these out. Secondly, due to misspellings, some categories may not be included in the desired group and you will also have to sort these out. But you can exclude the properly matched responses as you identify the remaining problematic cases. So, in short, there is no one line solution to these kinds of problems.

. list other if infrastructure , sep(10)

     +---------------------------------------------+
     |                                       other |
     |---------------------------------------------|
  3. | need improvememt in the road infrastructure |
  4. |      road infrastructure improvement needed |
  6. |      needed road infrastructure improvement |
  7. |      needed road infrastructure improvement |
  8. |             road infrastructure improvement |
 14. |             road infrastructure improvement |
 16. |                      poor druk air services |
     +---------------------------------------------+

. list other if business, sep(10)

     +-------------------------------------------------------------------------------------------------------------------------------+
     | other                                                                                                                         |
     |-------------------------------------------------------------------------------------------------------------------------------|
 13. | not getting loan for rural areas macro trade business provide low interest that will benefits for their business              |
 15. | location problem and facing problem to pay the interest for the business                                                      |
 17. | if the business items sales in same rate then it would be benifites to the shopkeeper                                         |
 20. | if government strict to bar licence those given on hair its effect on hairing owner business                                  |
 21. | problem of space for business purposes                                                                                        |
 22. | tax rates is qute not equal and itz unfair                                                                                    |
 23. | government to allow few nonbhutanese for sawmill works to get experience for bhutanese workers if government grant on half .. |
 25. | renewal rate for snooker is quite high compare to other business like general shop                                            |
 26. | business loan if government issue than it be benefits                                                                         |
     +-------------------------------------------------------------------------------------------------------------------------------+