View source for R Native API meeting 2016-06-30

Informal meeting after the end of the useR! 2016 conference.

Participants: Michael Sannella, Torsten Hothorn, Dirk Eddelbuettel, Karl Millar, Simon Urbanek, Mick Jordan, Lukas Stadler

Discussion topics:
* (Dirk) From the POV of Rcpp, lots of useful functionality is hidden and not part of the official API. It hasn't changed in a long time, why not make available?
:* It's not uncommon that people copy out code to make it available.
:* comment on data.table: it has a tiny dependency trail, and keeps working with very old R versions.
* (Torsten) Packages like stats do not export their functionality at the native level (or there are problems with dependency resolution).
:* Another case where people start copying out code.
:* Is it possible to get symbols from specific package? yes...
* "eval" could be much more efficient if it had a "prepare" and an "execute" step, like prepared DB statements.
:* Combined with a concise API, this would allow much more R functions to be reused on the native side, without a need for explicit C API.
::* Or have simple C wrappers, which can be replaced with a direct implementation in case of performance problems.
::* Do connection functions, e.g., have to be efficient?
::* Makes for good documentation - "behaves like as.integer" (maybe "sans S3/S4 dispatch")
* Is it "future proofing the API" or "future proofing packages"?
* Discussions related to CRAN:
:* Abandoned but popular packages sometimes get fixed by CRAN maintainers.
:* How could a larger set of changes produced by API renamings be handled?
::* Hard in the current system...
::* Having "master" versions of all packages on github would help.
::* Licensing / openness concerns with github?
:* Testing of GNUR with modified API?
::* Many packages require additional steps, installed libraries, etc.
::* Maybe r-hub could help? (Lukas will contact Gabor Csardi)
::* Two levels where changes can cause packages to fail: installing (compiling) and testing (where examples exist)
* What's the reason for the different prefixes?
:* Rf_..., R..., or no prefix, camel case, upper case, underscores, etc.
:* Historical reasons - cleanup could be done with tools or sed scripts.
* USE_RINTERNALS does two things: additional functionality and better performance
:* the former could be achieved by different include files
:* the latter should not be necessary (why not have everything at top speed, but leave the API in a state that can be verified?)
:* it should be possible to create a wrapper around the API that checks the (documented) contract as tightly as possible
* The manual still explains functionality that is generally considered to be wrong (e.g., "TYPE(x) = LANGSXP;")
* There should be no global variables, only functions (or at least a contract that allows them to be implemented as functions)
:* Not only CRAN - we need to describe the universe of (important?) packages.
:* Dependencies between functions? (sic!)

* General steps this WG should/could take:
:* Tighten API - remove stuff that is not used
::* Remove altogether, or deprecate (or hide behind a #define USE_DEPRECATED_API)
:* Renaming functions?
::* Maybe we want to introduce a new naming scheme?
::* Maybe have a period with both naming schemes
:* Document the functions
::* Describe the arguments and its contract.
::* Who could do that? For some functions only core R developers can give a real account of their intended contract.
::* Some functions are tightly related to R functions - maybe describe them in relation to these?
:* Breaking packages is ok, to a certain degree
:* You could do a lot via eval if the details of its behavior were defined well and non-surprising
::* Getting proper error context at the C level?
::* Java solved this with the Java Virtual Machine Tooling API (JVMTI)
:* Maybe create shims of R functions as a new API? docs?

* Immediate next step:
:* (Lukas) Define the "tighten API" task, what it entails, as a (student?) project, and find a "volunteer"