[Maarten Buis]

There is a difference between github search and search. As far as I am concerned, if search cannot find it then it does not exist, and I am free to use that name as I like (with the usual restriction that StataCorp reserves certain words).

[haghish]

That is true, and is a problem...

I cannot change the search command to cover GitHub, of course. But I can develop a general search command (e.g. findall) that adds the results from GitHub to the search command and view the results. That'd be a module in itself, but I think it is important. Thanks for the point.

[haghish]

Maarten Buis
I wrote findall and in addition to adding the results of GitHub search to results of the search command, it also specifies the date of the latest release on each server, including SSC and GitHub. This would provide a clue that which server hosts the latest version.

You can install the new command using github install which also takes care of the dependencies:

github install haghish/findall

[Maarten Buis]

I appreciate what you try to do, but that does not solve the problem. What somebody else decides to write the findreallyall command that also looks in the FooHub repository, and a third person writes the findeverything command that looks in the BarHub repository (but forgets to look in the FooHub repository)? I am afraid that there is one and only one entity that is authoritative in this respect: StataCorp. If they choose to include GitHub in search (and they very well might do so) then GitHub exists as far as I am concerned, if they do not, then it does not exist.

[haghish]

I agree, but that's beyond my will. For now, I have done all I could and argued all I knew. The rest is up to the users.

There are already a few Stata packages on GitHub that are not available elsewhere and probably the number will be growing. The least is that findall can avoid some of the problems, allowing the users also to explore GitHub (at least for installing software) and get the release dates of packages. Whether StataCorp wants to implement the search command for GitHub is not something I can decide, but it'd be very nice to have that feature indeed.

[wbuchanan]

The one thing I would say is that if the develop already needs to create the package and TOC files, it would be just as easy for the user-programmer to create a new branch of the repository with the name gh-pages and/or use the static site generator from GitHub and deliver the package to Stata users via

net install mypackage, from("http://username.github.io/repoName")

. This also allows additional tutorials/materials to be publicly posted in a way that could be useful for others but would not necessarily be a part of the installed package.

[haghish]

Indeed. making project website is so simple with GitHub and the bare minimum is that GitHub allows ou to load the README.md file from the repository and turn it to a website! Here is an example from the github repository https://haghish.github.io/github/

Billy I guess it should be from("https://username.github.io/repoName")?

[wbuchanan]

It would be safer and avoid installation problems to use:

net inst github, from("http://haghish.github.io/github")

Calls made to URLs over HTTPS spin up the JVM. If you're distributing source that also includes compiled Java binaries for plugins the HTTPS calls will lock the classpath and force the user to restart Stata in order to use the package. Using HTTP avoids that.

[haghish]

It would be safer and avoid installation problems to use:

net inst github, from("http://haghish.github.io/github")

I can't install it. My laptop (OS X 10.10.5, Stata 14.2) only installs it from https, i.e.

net inst github, from("https://haghish.github.io/github")

and for your suggested code returns an error saying:

. net inst github, from("http://haghish.github.io/github")
file http://haghish.github.io/github/stata.toc not found
server says file permanently redirected to https://haghish.github.io/github/stata.toc
http://haghish.github.io/github/ either
  1)  is not a valid URL, or
  2)  could not be contacted, or
  3)  is not a Stata download site (has no stata.toc file).

current site is still http://fmwww.bc.edu/RePEc/bocode/b/
r(601);

Does it work on your machine?

[James Hassell (StataCorp)]

It would be safer and avoid installation problems to use:

net inst github, from("http://haghish.github.io/github")

Calls made to URLs over HTTPS spin up the JVM. If you're distributing source that also includes compiled Java binaries for plugins the HTTPS calls will lock the classpath and force the user to restart Stata in order to use the package. Using HTTP avoids that.

William is correct that the Java classpath gets initialized when the Java Virtual Machine is initialized and any Java libraries that are added after that, will not be found. We hope to improve that behavior, but any change will need to be done under version control and introduced later.

[wbuchanan]

haghish I was just using that as an example. To get things set up quickly you can use:

! git branch gh-pages
! git checkout gh-pages
! git commit -m "Created simple example for gh-pages branch"
! git push -u origin gh-pages

This assumes that you've set up origin as the alias to the remote repository location. I use a different approach myself and you can also create the branch on GitHub, pull the changes into your local copy of the repository, merge the content from the master branch, then push back to the gh-pages branch on the GitHub server. That's how I distribute several of my packages now since it centralizes things a bit more and provides bit better analytic support than using my own web server.

[haghish]

Update on the github command:

The biggest challenge with using GitHub has been creating the pkg and toc files, which are lacking in the majority of the repositories. The new release of the github command allows installing repositories that lack these files as well, using the force option.

And it seems you can update github using itself.

github install haghish/github

or if you don't have the command installed:

net inst github , from("https://haghish.github.io/github")

[Nick Cox]

It's your problem and project. but that strikes me as very possibly the wrong way to go.

It's utterly standard advice ([P] net) that packages installed for use by others should have .pkg and .toc files. It is alarming to infer -- I may well be misreading this -- that there may be programmers who don't know this, but that's another story. Quite possibly their stance is that if you use GitHub then you take responsibility for your own installations. But those who live outside the law must take the consequences. (I do that too in (e.g.) using existing English words for some program names, contrary to explicit StataCorp advice, and expect the consequence that new Stata commands may break those programs.)

If your hope is that StataCorp in the long run include support for GitHub, then incompatibilities with the Stata standard are not helpful, and indulging them through work-arounds is at best a short-term fudge.

I have to guess that there is no way that StataCorp will abandon that standard or include work-arounds for packages that are incomplete. Indeed, a package cannot be recognised as such by Stata without such files!

At the same time, here is another wild guess, and naturally I do not speak for StataCorp: nothing non-trivial will now be added to the development list for Stata 15. I don't know more than I am saying, but am just drawing on the known habits of StataCorp in working towards previous releases.

[haghish]

Jesse Wursten I can also develop a very bad option for github command to install the files of the repository even if there is no toc and pkg file (i.e. create them on the fly). I call it a bad option because it is not disciplined enough and very casual. But it can work just fine for users who are not familiar with GitHub. Actually, most of the information stated in these two files can be obtained from the repository (description, release date, list of files, author, etc...). So yeah, I can make this happen, although experienced users won't favor this, I assume.

Nick I totally agree and I mentioned it before myself; see the quote.

Most users are not aware of the fact that they need the pkg and toc, which it is a pitty. Until now, only those who hosted their packages on private or university servers needed to create these files, which makes a perfect sense why the majority are unaware of them. For now, they can also install their software from GitHub, only if they force it. However, the repository documentation makes a clear recommendation to build the pkg and toc files and avoid the force option. I also add thatgithub search favors installable packages, regardless of the package hits score, which I hope acts as an incentive for the Stata GitHub users to include these files in the repository. But I am afraid it takes time.

About the last point, unfortunately I am unaware of StataCorp policies. But if StataCorp decides to implement or contribute to findall or github packages, or make any comment, I'll be very cooperative to implement their suggestions. Until now my only intention was making searching and installing from GitHub convenient.

[wbuchanan]

haghish I actually se things as being a bit more problematic. How would you avoid installing user written code that was never intended to be used as a callable command? How would you differentiate between Stata scripts, programs, etc.... What if people intend to have other material distributed with their package? The work around you proposed effectively strips end users of their ability to make that decision if they use GitHub to host a remote Git repository of their work. While I think tools/encouragement to use version control systems is great, this is a case where there could potentially be more harm than good that comes from it since some users may now be reluctant to use the platform knowing that others might install things that they are working on prior to being release ready.