Different Regression Results after each rerun

[Pauline Mueller]

Hello everbody,
excited to make my very first post. I am currently writing my Thesis with Stata. I am texting here, because obviously in my code is somehow a mistake, as my regressions do not produce constant results.
I am using xtreg with fixed effects and clustered standards errors. The Problem is that it seems like, Stata takes randomly observations and change the amount of observations and clusters. I already searched for duplicates, I tested my merge command (1:1) and put after the sort comment "stable". But still the regressions are not constant. Anywhere in my code is probably room for randomness.. But I couldn't find it, I already investigated step by step but without any solution. I am investigating in the research setting of platforms on a platform-month level and want to know if 1) marketshare positively influences quality of complements on the platform and 2) if an increased competition among complementors negatively influence the quality of complements.

You see the code below, and I would be incredible thankful if you have a guess which might help! Thank you a lot!


**Import**

(... Here I just aggregated the the single observations from each platform to a total month observation to do the reshape later)


//Reshape data wide to long format
bys platform: keep if _n == 1
reshape long Revenue Units , i(platform) j(month) string



//Only keep a few platforms
keep if inlist(platform, "snes", "genesis", "jaguar", "n64", "3do", "playstation") | inlist(platform, "dreamcast", "gamecube", "ps2", "xbox", "ps3", "wii", "xbox360")




**Definition of platform success factors H1
//1 Generating a date variable out of the string month
split month, p("_")
drop month
rename month2 year

gen month = 1 if month1 == "Jan"
replace month = 2 if month1 == "Feb"
replace month = 3 if month1 == "Mrz"
replace month = 4 if month1 == "Apr"
replace month = 5 if month1 == "Mai"
replace month = 6 if month1 == "Jun"
replace month = 7 if month1 == "Jul"
replace month = 8 if month1 == "Aug"
replace month = 9 if month1 == "Sep"
replace month = 10 if month1 == "Okt"
replace month = 11 if month1 == "Nov"
replace month = 12 if month1 == "Dez"

gen year2 = real(year)
drop year
rename year2 year




//Between platform competition independent variable
//REVENUE marketshare within generation
gen generation = 4 if platform == "snes" | platform == "genesis"
replace generation = 5 if platform == "jaguar" | platform == "n64" | platform == "3do" | platform == "playstation"
replace generation = 6 if platform == "dreamcast" | platform == "gamecube" | platform == "ps2" | platform == "xbox"
replace generation = 7 if platform == "ps3" | platform == "wii" | platform == "xbox360"

sort generation year month
by generation year month: egen total_units_generation = total(Units)
by generation year month: egen total_revenue_generation = total(Revenue)

gen marketshare_units_gen = Units/total_units_generation
replace marketshare_units_gen = 0 if marketshare_units_gen ==.

gen marketshare_revenue_gen = Revenue/total_revenue_generation
replace marketshare_revenue_gen = 0 if marketshare_revenue_gen ==.

//Generate Dummies to test U shape
gen market_bins = 1 if marketshare_revenue_gen <= 0.2
replace market_bins = 2 if marketshare_revenue_gen > 0.2 & marketshare_revenue_gen <= 0.4
replace market_bins = 3 if marketshare_revenue_gen > 0.4 & marketshare_revenue_gen <= 0.6
replace market_bins = 4 if marketshare_revenue_gen > 0.6 & marketshare_revenue_gen <= 0.8
replace market_bins = 5 if marketshare_revenue_gen > 0.8



sort platform year month
//Controls
//1 Platform price
sort platform year month
gen platformprice = Revenue/Units
gen lnplatformprice = ln(platformprice)


//2 Platform age
// generating a numeric variable to calculate the platform age later
generate intromonth_str = string(intromonth, "%tm")
split intromonth_str, p("m")
drop intromonth_str
drop introyear intromonth
rename intromonth_str1 introyear
rename intromonth_str2 intromonth

gen introyear2 = real(introyear)
drop introyear
rename introyear2 introyear

gen intromonth2 = real(intromonth)
drop intromonth
rename intromonth2 intromonth

sort platform year introyear
gen periodyear = (year - introyear)*12
sort platform month intromonth
gen periodmonth = (intromonth - month)
gen age_platform = periodyear + periodmonth if periodyear >=0
replace age_platform =. if age_platform < 0


sort platform year month
bys platform year month: keep if _n==1
//Merge Gamelevel Dataset
merge 1:1 platform year month using "/.../Thesis/Gamelevel_input.dta"
//drop if no information on console & drop if console is not active anymore & drop if platform has no competitor
drop if _merge == 1 & newcomplements_platform ==.
drop if _merge == 2
drop if Revenue ==.
drop if Revenue == 0
drop if Units < 1000
drop if marketshare_revenue_gen == 1
drop if marketshare_revenue_gen == 0
//generate platform_id and period for panel
sort year month
egen period1 = group(year month)
sort platform year month
egen platform_id = group(platform)




sort platform year month, stable
**H2b Distinctive positioning
//proportion of games offered by a platform in genre j at time t
gen prop_action = cum_games_platformt_action /cum_games_platform
gen prop_adventure = cum_games_platformt_adventure/cum_games_platform
gen prop_arcade = cum_games_platformt_arcade/cum_games_platform
gen prop_children = cum_games_platformt_children/cum_games_platform
gen prop_family = cum_games_platformt_family/cum_games_platform
gen prop_fighting = cum_games_platformt_fighting/cum_games_platform
gen prop_flight = cum_games_platformt_flight/cum_games_platform
gen prop_other = cum_games_platformt_other/cum_games_platform
gen prop_racing = cum_games_platformt_racing/cum_games_platform
gen prop_roleplay = cum_games_platformt_roleplay/cum_games_platform
gen prop_shooter = cum_games_platformt_shooter/cum_games_platform
gen prop_sport = cum_games_platformt_sport/cum_games_platform
gen prop_strategy = cum_games_platformt_strategy/cum_games_platform


//gen distribution of gamegenres per platform for descriptive table similar to Cennamo & Santalo Table 1:
bysort platform: egen mean_platform_action = mean(prop_action) if id_games > 0
bysort platform: egen mean_platform_adventure = mean(prop_adventure) if id_games > 0
bysort platform: egen mean_platform_arcade = mean(prop_arcade) if id_games > 0
bysort platform: egen mean_platform_children = mean(prop_children) if id_games > 0
bysort platform: egen mean_platform_family = mean(prop_family) if id_games > 0
bysort platform: egen mean_platform_fighting = mean(prop_fighting) if id_games > 0
bysort platform: egen mean_platform_flight = mean(prop_flight) if id_games > 0
bysort platform: egen mean_platform_other = mean(prop_other) if id_games > 0
bysort platform: egen mean_platform_racing = mean(prop_racing) if id_games > 0
bysort platform: egen mean_platform_roleplay = mean(prop_roleplay) if id_games > 0
bysort platform: egen mean_platform_shooter = mean(prop_shooter) if id_games > 0
bysort platform: egen mean_platform_sport = mean(prop_sport) if id_games > 0
bysort platform: egen mean_platform_strategy = mean(prop_strategy) if id_games > 0


sort platform generation year month, stable
//mean propotion of games offered by each generation in genre j at time t
//gen4
gen prop_action_gen_mean = mean_gen_action/cum_games_generation
gen prop_adventure_gen_mean = mean_gen_adventure/cum_games_generation
gen prop_arcade_gen_mean = mean_gen_arcade/cum_games_generation
gen prop_children_gen_mean = mean_gen_children/cum_games_generation
gen prop_family_gen_mean = mean_gen_family/cum_games_generation
gen prop_fighting_gen_mean = mean_gen_fighting/cum_games_generation
gen prop_flight_gen_mean = mean_gen_flight/cum_games_generation
gen prop_other_gen_mean = mean_gen_other/cum_games_generation
gen prop_racing_gen_mean = mean_gen_racing/cum_games_generation
gen prop_roleplay_gen_mean = mean_gen_roleplay/cum_games_generation
gen prop_shooter_gen_mean = mean_gen_shooter/cum_games_generation
gen prop_sport_gen_mean = mean_gen_sport/cum_games_generation
gen prop_strategy_gen_mean = mean_gen_strategy/cum_games_generation

//Deviation platform from the mean --> distinctive positioning
//Deviation focal platform from mean value of generation
gen dev_action = (abs(prop_action - prop_action_gen_mean))
gen dev_adventure = (abs(prop_adventure - prop_adventure_gen_mean))
gen dev_arcade = (abs(prop_arcade - prop_arcade_gen_mean))
gen dev_children = (abs(prop_children - prop_children_gen_mean))
gen dev_family = (abs(prop_family - prop_family_gen_mean))
gen dev_fighting = (abs(prop_fighting - prop_fighting_gen_mean))
gen dev_flight = (abs(prop_flight - prop_flight_gen_mean))
gen dev_other = (abs(prop_other - prop_other_gen_mean))
gen dev_racing = (abs(prop_racing - prop_racing_gen_mean))
gen dev_roleplay = (abs(prop_roleplay - prop_roleplay_gen_mean))
gen dev_shooter = (abs(prop_shooter - prop_shooter_gen_mean))
gen dev_sport = (abs(prop_sport - prop_sport_gen_mean))
gen dev_strategy = (abs(prop_strategy - prop_strategy_gen_mean))

// sum deviations over the genres = Distinctiveness
egen distinctiveness = rowtotal(dev_action dev_adventure dev_children dev_arcade dev_family dev_fighting dev_fighting dev_other dev_racing dev_roleplay dev_shooter dev_sport dev_strategy)

//Interaction term distinctiveness & #newcomplements_platform
gen Int_distinct_newcomp = distinctiveness * ln_cum_games_platform

sort platform year month, stable
*set dataset to panel


xtset platform_id period1

[Richard Williams]

Sometimes you use the stable option, sometimes you don't, and you also use the bysort command which does not use the stable option.

Instead of bysort, you could say

sort whatever, stable
by whatever: ...

Also add the stable option to every sort command.

I would also add something like

set seed 125789

at the top of the ado file.

My guess is that if you do that you will get consistent results.

Having said that -- if you have to use the stable option, you may be doing something wrong. See this thread, in particular Maarten Buis's post #3:

https://www.statalist.org/forums/for...ing-3-datasets

Among other things, Maarten says "I also suspect -sort- is the problem. But I don't think the -stable- option will solve it. Instead I think the -stable- option will just hide the problem with the identifying variables. So I would strongly recommend not to use the -stable- option. Instead she should take a close look at the identifying variables and find the real problem." He also offers some sources to check.

So, you can try my suggestions for stable but if it does seem to "solve" the problem, it may be that there is another problem you are missing.