a master do-file to make changes in other do-files?

[Nicola Cenacchi]

Hi all,

I am fairly new to Stata, so I often have to find solutions to new "needs".
I have a question I have been unable to answer yet (especially point 2).
Is it possible to write a Master do-file that would:

1. open do-file X
2. replace several words in dofile X with different words
3. save dofile X as dofile X1
4. run dofile X1

many thanks
Nicola

[daniel klein]

Yes, it is possible. But it sounds like do-file X should probably accept arguments and/or should better be a program. It is hard to be more specific without a minimal example.

[Nicola Cenacchi]

Thanks Daniel,

If I understand correctly you would need to see examples of my dofiles.
I have taken two of them, pared them down, and pasted them below. In each of them there are some words I need to change.
For instance in the first dofile-example "ssp270" needs to be replaced with "ssp585", and in the second example wherever it reads 541, it needs to be replaced with 563.

I hope this helps
thanks
Nicola

Example 1

clear
capture log close
set more off

log using MASTER_ar6.log, replace

insheet using "AR6_ssp270.csv", comma

rename val Value
tostring year, replace

*Production and Demand
replace impactparameter="Total Area" if impactparameter=="TAreaXAgg -- Total Area"
replace impactparameter="Area" if impactparameter=="AreaXAgg -- Area"
replace impactparameter="Production" if impactparameter=="QSupXAgg -- Commodity Supply"
replace impactparameter="Total Yield" if impactparameter=="TYldXAgg -- Total Yield"
replace impactparameter="Yield" if impactparameter=="YldXAgg -- Yield"
replace impactparameter="Animal Numbers" if impactparameter=="AnmlNumXAgg -- Animal Numbers"
replace impactparameter="Animal Yield" if impactparameter=="AnmlYldXAgg -- Animal Yield"
replace impactparameter="Total Demand" if impactparameter=="QDXAgg -- Total Demand"
replace impactparameter="Food Demand" if impactparameter=="QFXAgg -- Household Demand"
replace impactparameter="Feed Demand" if impactparameter=="QLXAgg -- Livestock Feed Demand"

save all_data, replace

outsheet using "Extracted_ExcelFiles\\Total_Production_ar6.csv"         ///
                                              if (impactparameter=="Total Area" |     ///
                                              impactparameter=="Total Yield" |     ///
                                              impactparameter=="Total Demand" | ///
                                              impactparameter=="Food demand" | ///
                                              impactparameter=="Production"), comma replace                                              

                                                               

clear

log close
set more on

exit

And a second example below

clear
capture log close
set more off

log using AR6_food_security_ssp270.log, replace

insheet using "Food security_ar6.csv", comma

tostring year, replace
drop if impactparameter=="Kcal pc by Commodity"
drop productiontype
drop commodity

replace impactparameter="pcKcal" if impactparameter=="Kcal pc"
replace impactparameter="PopAtRisk" if impactparameter=="Pop at risk of hunger"  
replace impactparameter="ShareAtRisk" if impactparameter=="Share Pop at risk of hunger"
replace impactparameter="Undernourished" if impactparameter=="Malnourished Children"

*-----*FILE #1 %diff in 2030 and 2050 but for each single GCM and keeping all regions and cty

reshape wide value, i(impactparameter region year) j(scenario) string

save wide_allGCMs, replace
drop if valueREFar6_NoCC==.

local scenarios valueREFar6_NoCC valueGFDL379 valueGFDL541 valueUKESM379 valueUKESM541 valueMRI379 valueMRI541 valueMPI379 valueMPI541 valueIPSL379 valueIPSL541

foreach scenario in `scenarios'{
    
    gen pd_`scenario'=(`scenario'- valueREFar6_NoCC)/valueREFar6_NoCC
    
}


/*then I can drop all the variables starting with yield
getting rid of variables I no longer need*/
drop value*

*percent diff is done so I can now drop the base
drop pd_valueREFar6_NoCC

save wide3, replace

reshape long pd_, i(impactparameter region year) j(scenario) string
rename pd_ perdiff

*now I need to create a co2 variable to keep track of 379 and 541 in a different way
gen co2="379"
replace co2="541" if strpos(scenario,"541")
order co2, after(impactparameter)

*I need to rename the scenarios - the co2 variable allows me to distinguish between 379 and 541, so the names can be generalized
replace scenario="GFDL" if scenario=="valueGFDL379" | scenario=="valueGFDL541"
replace scenario="IPSL" if scenario=="valueIPSL379" | scenario=="valueIPSL541"
replace scenario="MRI" if scenario=="valueMRI379" | scenario=="valueMRI541"
replace scenario="MPI" if scenario=="valueMPI379" | scenario=="valueMPI541"
replace scenario="UKESM" if scenario=="valueUKESM379" | scenario=="valueUKESM541"

save file1_foodsecurity_ar6_GCMs_pd, replace
outsheet using "file1_foodsecurity_ar6_GCMs_pd.csv", comma replace

clear

log close
set more on

exit

[Maarten Buis]

Do files accept arguments. See https://maartenbuis.nl/workshops/sta...l#slide16.smcl

[Nicola Cenacchi]

Thanks Maarten, but I am afraid I do not know how to apply that to my question

[daniel klein]

So you can do something like this:

//  -------------------------  begin do-file_1
args ssp_number

(code omitted)

insheet using "AR6_ssp`ssp_number'.csv", comma

(code omitted)
//  ---------------------------  end do-file_1

When you then call the do-file, call it as

do "do-file_1" 270

or

do "do-file_1" 585

depending on which ssp# you want.

The core idea here is to write do-files once but write them in a way that handles variable inputs (i.e., arguments).

[Nicola Cenacchi]

Hi Daniel,

thank you very much for suggestion. A question ... I am not sure I understand in the code above why it says:

 insheet using "AR6_ssp`ssp_number'.csv", comma

I guess it should be something like:  
 insheet using "AR6_ssp`arg'.csv", comma

If I understand correctly your comment, the idea is to write a single dofile, with one or more stand-ins for numbers or text that you want to replace, which is why I am not entirely understanding the code.

Also, is there a way to have more than one set of arguments? something like

args1 270 585 etc
args2 apple tree moon etc

many thanks
Nicola

[daniel klein]

Maartens' presentation briefly introduces the core concept. If that does not suffice, please read [U] 16.4 Programming with do-files for the full story; I do not think that I can do a better job in a post here than the manual does.

[Maarten Buis]

in the code above why it says:

insheet using "AR6_ssp`ssp_number'.csv", comma

I guess it should be something like:

insheet using "AR6_ssp`arg'.csv", comma

No, Daniel's code is correct.

You have a main .do file that contains this line

// ------------------------------ begin main.do 
 do "do-file_1" 270 
//------------------------------ end main.do

and do_file_1.do contains the following lines:

// ------------------------------ begin do_file_1.do
args ssp_number

(code omitted)

insheet using "AR6_ssp`ssp_number'.csv", comma

(code omitted)
//------------------------------- end do_file_1.do

Lets step through what happens when you run main.do:

• It runs the do-file do_file_1.do and passes the argument 270 to it
• This means that when the do-file do_file_1.do starts running, a local macro called 1 will be created containing 270 (if we passed two arguments the local macros 1 and 2 would be created, etc)
• `1' is not a very informative name for a macro, so Daniel started his do_file_1.do file with the command args ssp_number. This renames the local macro 1 to ssp_number, which still contains the number 270
• Then it contains the command insheet using "AR6_ssp`ssp_number'.csv", comma. This contains a local macro (ssp_number), so before executing that command Stata first replaces the local macro with its contents: so Stata executes the command: insheet using "AR6_ssp270.csv", comma

The power of this strategy is that you can pass different number to it, so main.do could contain something like

// ------------------------------ begin main.do
do "do-file_1" 270
do "do-file_1" 585
do "do-file_1" 321
//------------------------------ end main.do

or

// ------------------------------ begin main.do
local ssp_nums = "270 585 321"
foreach ssp_num of local ssp_nums {
    do "do-file_1" `ssp_num'
}
//------------------------------ end main.do

Also, is there a way to have more than one set of arguments? something like

args1 270 585 etc
args2 apple tree moon etc

Yes you can pass multiple arguments to a .do file. For example, your do_file_1.do could have something like this:

// ------------------------------ begin do_file_1.do
args ssp_number fruit

(code omitted)

insheet using "AR6_ssp`ssp_number'.csv", comma

display "have you eaten a `fruit' today?"

(code omitted)
//------------------------------- end do_file_1.do

So now do_file_1.do expects two arguments, which will start as local macros named `1' and `2', but the command args renames the local macro 1 to ssp_number, and local macro named 2 to fruit. Your main.do could than look something like this:

// ------------------------------ begin main.do
do "do-file_1" 270 apple
do "do-file_1" 270 orange
do "do-file_1" 270 lemon
do "do-file_1" 585 apple
do "do-file_1" 585 orange
do "do-file_1" 585 lemon
do "do-file_1" 321 apple
do "do-file_1" 321 orange
do "do-file_1" 321 lemon
//------------------------------ end main.do

or this:

// ------------------------------ begin main.do
local ssp_nums = "270 585 321"
local fruits = "apple orange lemon"
foreach ssp_num of local ssp_nums {
    foreach fruit of local fruits {
        do "do_file_1" `ssp_num' `fruit'
    }
}
//------------------------------ end main.do