There has been a lot of discussion on the interweb about the support for legislation in the US by the major academic publishers that would place stifling restrictions on access to Government-funded science outputs published in their journals. In effect, the Research Works Act would hand these same wealthy publishers a licence to print money on the back of tax payer-funded research and the tax payer-funded efforts of the scientists required to peer review the manuscripts. We the scientists do the research and peer review the manuscripts, the tax payer pays for all of this and yet the academic publishers charge upwards of £30 to access a single article and hundreds to tens of thousands of pounds for individuals and institutions to subscribe to their journals. All the publishers add is

1. the infrastructure that allows them to track papers in peer review (which is largely web-based and rubbish),
2. the typesetting of manuscripts into final journal style form, and
3. the websites and infrastructure allowing people to get access to journals electronically (for the proper fee of course).

The major publishers make 20-30% 30-40% profit-margins on their revenues yet contribute little.

And yet they want more…! I’m not suggesting that the publishers should add their value without monetary reward, but they are being disingenuous when they claim to be adding significant value, which they need to recoup, to the manuscripts we supply.

So from today I am on strike when it comes to performing peer-review for journals from the stables of the major academic publishers. I will only review for journals that release all papers under Open Access frameworks. As such, I will now only be submitting papers to Open Access-only journals.

My first action was to decline a review request that came in a day or two ago. I was waiting on a response from John Wiley in the US before deciding whether to accept the invite or not, but as none was forthcoming I emailed the handling editor this morning to decline. Below is the email I sent setting out my reasons for my action.

Dear XXXXX,

Apologies for the delay in replying. I was waiting on a response from
John Wiley to some queries I had about their position with regard to the
Research Works Act (RWA) in the US and Open Access to research papers in
general. I am very concerned that large academic publishers are actively
pursuing an agenda that will severely restrict the ability for
scientists around the world to conduct science in an environment of open
exchange of ideas and results.

In particular the RWA would if passed give these publishers effective
copyright over the final published record of federally-funded research,
allowing them rights to charge exorbitant fees for access to these
papers and journals, further deepening their already deep pockets.

Having not received any form of response from John Wiley in the US I am
now refusing to perform peer review of papers submitted to their
journals to demonstrate my general disgust at their support for stifling
open access to scientific papers. These publishers rely upon the free
(to them, but not to the tax payers that pay the salaries of scientists
doing the reviews) provision of peer review and I will no longer perform
such duties whilst the publishers make huge profits (upwards of 30% of
revenues) feeding off our generosity.As such I must respectfully decline
your invitation to review manuscript Ms# o20360.

Please feel free to pass this message on to Prof. XXXXX XXXXX, XXXXX
Editor in Chief, and your publisher should you so desire.

Yours,

Dr. Gavin Simpson

Not sure what effect, if any, this will have; I am but a small fish in a very big pond. If more of my colleagues adopt similar strategies to handling peer review requests from the major publishers perhaps they will get the message.

I’ll let you know how things progress and post any response I get from the publisher or the editor of the journal in question.

Update: The managing editor of the journal in question has replied to say that they will be passing my comments on to the journal editor-in-chief and the publisher. Will let you know if I hear anything further.

0.8-0 is a moderate update to analogue containing some new functionality, some of which is there for testing/experimentation (like the fancy principal components regression). The main user visible changes are:

• crossval() new function to perform leave-one-out, k-fold, n k-fold, and bootstrap cross-validation on transfer function models. A method for wa() models is provided.
• pcr() performs principal components regression. Designed to allow transformations in the spirit of Legendre & Gallagher (2001, Oecologia) that allow PCA to be usefully applied to species data.
• varExpl() and gradientDist() are two new functions that extract the amount or variance explained by ordinations axes and the distances or locations along ordination axes. Methods currently available for cca() and prcurve() objects.
• weightedCor() implements one of the tests from Telford & Birks (2011, QSR) based on the weighted correlation of WA optima and constrained ordination species scores.
• Stratiplot() now handles absolute data better following a few bug fixes and general improvements in the underlying code. panel.Stratiplot() gains new arguments gridh and gridv to allow user control of the grid lines on panel if plotted.
• mat() gains a new argument `kmax` which can be used to limit the number of analogues considered as models when fitting MAT transfer functions. By default, mat() considers models with 1 through to n-1 analogues (n = number of sites). kmax can control this upper limit which will speed up fitting models, especially for large training sets. Invariably one wouldn’t want to average over entire training sets to produce predictions, or even over large numbers of analogues.

There were also many bug fixes and minor enhancements. Full details can be found in the ChangeLog, the relevant portion of which is appended below. Several development releases were made on R-forge after the 0.7-0 release to CRAN. These development versions were not publicly released, but the changes they implemented are all present in 0.8-0 of analogue.

Version 0.8-0

	* Updated Example test checks and packaged for release to CRAN
	Jan 11, 2012.

Version 0.7-7

	* mat: new argument `kmax` can be used to limit the number of
	analogues considered as models when fitting MAT transfer
	functions. By default, `mat()` considers models with 1 through
	to n-1 analogues (n = number of sites). `kmax` can control this
	upper limit which will speed up fitting models, especially for
	large training sets. Invariably one wouldn't want to average
	over entire training sets to produce predictions, or even over
	large numbers of analogues. As such I may set an upper limit for
	the default value of `kmax` before this is released to CRAN.

	* cumWmean, cummean: as a result of the above addition of `kmax`,
	these two functions now take a `kmax` argument also. The default
	behaviour is unchanged however.

	* chooseTaxa: `type = "OR"` was not working due to a typo. It
	returned the same as `type = "AND"`.

Version 0.7-6

	* Stratiplot: Handling of absolute data types was broken. Fix
	applied that should allow this to work if there are only
	absolute scale variables or a mix or relative and absolute
	data. All reletaive data should be unaffected.

	* panel.Stratiplot: gains arguments `gridh` and gridv` which
	control the number of horizontal and vertical grid lines used
	on each panel. These correspond to the `h` and `v` arguments of
	`panel.grid` in the Lattice package. The default is `-1` for
	both, which attempts to align the grid lines with the tick marks.

Version 0.7-5

	* weightedCor: implements one of the tests from Telford & Birks
	(2011, QSR) based on the weighted correlation of WA optima and
	constrained ordination species scores. Has a plot method.

	* rdaFit: Non-user (currently) function that implements RDA
	without all of the overhead of vegan::rda. As such it doesn't
	compute PCA axes and does not return all the components described
	by ?cca.object in package vegan. This function is used principally
	in weightedCor(). Has a scores() method. rdaFit() is not
	documented as the exact details of the function and its
	capabilities remain to be determined.

Version 0.7-4

	* gradientDist: new function to extract locations along an
	ordination axis. Methods for prcurve() and cca().

	* varExpl: new function to extract the amount of variance
	explained by ordination axes. Currently methods for prcurve() and
	cca() are available.

	* Namespace: analogue now has an explicit name space in
	preparation for R 2.14.0-to-be. Hence analogue now depends on
	Vegan >= 1.17-12.

Version 0.7-3

	* pcr: coef(), fitted(), residuals(), eigenvals(), performance(),
	and screeplot() methods added.

Version 0.7-2

	* pcr: new function pcr() performs principal components
	regression. Designed to allow transformations in the spirit of
	Legendre & Gallagher (2001) that allow PCA to be usefully
	applied to species data.

Version 0.7-1

	* crossval: new function to perform leave-one-out, k-fold,
	n k-fold, and bootstrap cross-validation on transfer function
	models. A method for wa() models is provided.
	* tests: package now has a test that the examples continue to
	return correct output.

The first talk I attended was on an initiative to map high latitude lakes and provide a data product for scientists. The mapping involved a new algorithm that appeared to perform far better when faced with satellite images of lakes during partial ice-out. The author mentioned that a new product for Alaska had been released a week or so ago and this will be very useful for a project proposal I am currently involved in writing.

The next talk, by a student of Stephen Carpenter and Michael Pace, was something I have been interested in for some time; identifying early warning indicators in ecological communities prior to regime shift. Several early warning indicators are invoked from theory of tipping points and regime shifts

• increased autocorrelation
• increased variance
• increased skewness
• conditional heteroskedasticty

The latter can be thought of as “autocorrelation” in the variance of the system. This indicator was demonstrated to be found in data from experimentally manipulated lakes where the fish community was altered. I got plenty of ideas from this talk, with a view to doing similar analyses on palaeoecological data.

I later spent a bit of time in a session focussed on the marine nitrogen cycle over the past few billion years. Not really my area of interest, but plenty of interesting ideas with regard to compound specific N isotope measurements, including information on compound specific pigment isotopes.

I went to a poster session on palaeo records of change in lake ecosystems. I met up with Dave Porinchu, someone I spent a week with in Helsinki at a Cladocera workshop 10 years or so ago. Consumption of a bottle of Talisker comes to mind. I met several other people here too; Yarrow Axford had an interesting poster on chironomid temperature reconstructions in a West Greenland lake for example and I chatted to her for a while about palaeoecological transfer functions, many of which I just don’t believe even though I teach the theory every year to plenty of palaeoecologists! I also bumped into Simon Goring, whom I’d had some email correspondence with a year ago regarding a new transfer function methodology he’d come up with. I joined him and some colleagues for a nice lunch in a local Thai restaurant.

Having had a two-course lunch, I didn’t need further sustenance and after an email session I headed back to my hotel and set to work on my river water temperature models, squashing an annoying data-processing bug. I set the laptop off data crunching through the multivariate smooth model fitting whilst I slept and now I have something to work with on the flight home so I can finish of my additive modelling paper for river water temperature trend modelling.

And that was day 3…

One of the problems of trying to blog a conference when you are going to session talks and socialising in the evening is keeping up to date with your blogging. Day 2 was a little slower in regard to talks and I was able to pick to some degree.

Sarah Hobbie gave a nice presentation on decomposition processes in soils and how reactive nitrogen affects decomp rates in leaf litter and soil organic carbon. She distinguished between slow and fast cycling pools of C in both leaf litter and soil organic matter. One of her key findings was that reactive N can increase initial decomp rates in litter but lead to great mass of C in the slow cycling carbon pool. Another key point made however was that exceptions to the various “rules” Hobbie presented were manifold in the literature and data.

I attended several other talks but this was more out of general interest and not something I intended making notes on. The Exhibitor floor opened for the first time on Tuesday so I spent a fair amount of time wandering around in looking at the various exhibits. Lunch consisted of an $8.50 ham and cheese sandwich from the stall in the Moscone Center (which wasn’t worth the money so won’t be eating there again!)

In the afternoon I attended a couple of talks on a communicating climate science theme. Mike Mann (of Penn State & Real Climate fame) gave an entertaining talk on the climate wars; dispatches from the front lines (and plugged his book of the same name due out on Jan — hmmm his advertising worked…). Whilst I agreed with the general points of engaging with climate sceptics in their own back yard (new and TV media) to counter their misinformation, I was left with the distinct feeling that had Mann and many of his colleagues engaged sooner and more fully with the skeptics, they’d have had an easier ride.At least they couldn’t have hit them once with the “they aren’t playing ball” outcry and have just been left with decrying them over the science, which Mann and colleagues could easily rebut. Openness in science is of prime importance; open data, open code, open methods. We shouldn’t have anything to hide, no matter how the data/results might be misinterpreted by opening up. We can always address misunderstanding or misinformation from the skeptics and do so in various media as suggested by Mann and do so in a way amenable to the lay person. Anyway, despite my reservations that Mann didn’t go far enough in his calls for engagement, his was a thoroughly entertaining and thought-provoking talk delivered to a pack auditorium.

After that I did some work and tended the email inbox, before heading off for a pizza and a beer with John Anderson in a small local Pizzeria, just up the hill from our hotel. Bed by 10pm. How Rock and Roll!

The day started off rather well for me as I got some good news about a research project proposal I am involved with within a minute of sitting down in Session B11F (Coupled Biogeochemical Cycles in Terrestrial and Aquatic Ecosystems I). Not quite 100% certain it will be funded as the funding body wants options for small budget cuts but looking very promising!

This (B11F) was a somewhat general session with several interesting (to me) talks. Two in particular piqued some interest and centred on methylmercury (MeHg) production and indicators of MeHg in a delta system. Of most interest was talk B11F-03 by Mitchell et al who had investigated the control of MeHg production by sulphur in a peatland setting by experimentally manipulating the system through the artificial addition and subsequent removal of sulphate deposition to look at the impacts of acid deposition on MeHg production and the lag times of recovery following reductions in deposition.

Brian McGlynn gave an interesting talk (B11F-08) on a methodological development for investigating nutrient uptake kinetics in stream ecosystems which allows the synergistic relationship between nutrients uptake kinetics to be modelled. The talk focussed on N and P stoichiometry in upland streams and their methodology allowed the authors to look at the effect N and P individually plus their combined uptake response. Their results showed that the three study streams were strongly co-limited by both nutrients. A talk later in the day by Tim Covino (B12D-04) went into greater detail on the methodology and an application.

The next session I attended was B12D Nitrogen Export from Headwater Catchments (I’ll just give the short session title — who comes up with these overly long titles anyway?) was particularly interesting to me given my nitrogen interests. Little of major consequence to my particular work (palaeo isotope geochemistry over the past few hundred years) but several talks covered topics relevant to the work Chris Curtis, Tim Heaton (British Geological Society, NIGL), Jan Kaiser (UEA) and I have been doing under the UK Defra Freshwater Umbrella programme on nitrate pathways and the fate of deposited N.

The session ended with a talk by Stephen Sebestyn which nicely covered much of the ground we’ve also been working on in the UK. Sebestyn covered a wider temporal range though, looking at when high atmospheric nitrate contributions to in-stream nitrate concentrations were observed (i.e. nitrate that had not been microbially processed); snow melt and rain on snow events in the main, plus some events in early spring before vegetation has kicked in, and a number of events in later summer and autumn. The take home message was that nitrate directly from atmospheric deposition could represent a large component of the in-stream nitrate (50+ %) but that there was large spatial heterogeneity in the proportion of atmospheric nitrate within small geographical regions.

At this point I headed out to the food court in the Westfield Mall for lunch, not by intention but I just happened to wander that way, along with thousands of fellow attendees. After grabbing a nice burger, fries and a Coke, I ended up sitting down at a table with a local gentleman. He explained he ate there every day, had done so for years, but had never seen so many people trying to get food down there.

After lunch I attended the session in which my talk had been put; H13G Digital Soil Mapping and Hydroecological Modelling. As the title suggests, the session was a mixed bag of talks resulting from the merging of several session proposals. I had been invited to talk in a session related to R and its use in modelling. My talk and one other from that session proposal made it into the final session and as it was now a much more general session I decided not to talk too much about R itself but to focus more on the modelling side of my work on river water temperatures. I can’t say too much about this work just yet; the funding body wants to do a press release when we get the work published in a journal and the paper has been in review at Hydrological Processes for four months and is currently sat on an editor’s desk awaiting a decision (four months is pretty poor turn around!) I have a paper in prep on the modelling methodology I used, all done in R of course(!), which I’ll write a blog post on when it is accepted (have to submit first). As a result of the merging etc, I wasn’t particularly academically interested in the talks in the session but I stayed throughout and saw some nice soil mapping and hydrological modelling presentations, and the session was reasonably well attended.

My talk went well, prompting a couple of questions and I managed to stick to time, which is good for me; I often talk slower when presenting and prefer more free-form slides than a tightly scripted set of bullets, but that does mean timings can be somewhat erratic!

The last session of the day that I attended was B14D on carbon isotopes and biomarkers. The biomarker scene is one I want to start getting into but on the nitrogen front looking at ¹⁵N in algal pigments and amino acids. Some interesting talks in here employing a level of chemistry that was way above my admittedly low level. Phil Meyers gave a nice talk on isoptopically light carbon in bulk organic matter in Phanerozoic black shales, discussing potential mechanisms that could give rise to the observed values (the deposits are marine in origin, but differ markedly from modern marine carbon isotope values for bulk OM). The entire session was part of the DIPPI-C initiative, which seems to be one to keep an eye on. The main thing I got from this session was the depths scientists are going to to understand what the isotopic measurements they make on organic compounds mean for the environment and depositional system that gave rise to the organic matter they study.

By now my day was pretty much done. I dealt with a bit of email and read through a paper I’m a co-author on, then met up for dinner with a couple of colleagues (John Anderson and Jasmine Saros) at an Italian/fusion restaurant.

Thankfully, I’d tired myself out so much that I only woke once during the night and was soundly asleep when the alarm kicked in at 0630 this morning. Seems like I getting this jet-lag licked!

The trip to San Francisco was uneventful if a bit tiring. I flew with Air Canada via Vancouver and the 10 odd hour flight there passed quite quickly (ably helped by a selection of mindless movies that suited my somewhat addled brain). The changeover at Vancouver was a reasonably pleasant experience, considering my past experiences of US immigration. I hadn’t realised this, but you go through immigration to the US at Vancouver, not at the final US destination, so transit passengers are corralled into a small waiting area until ones bag is retrieved and ready to head to the connecting flight. Once called up to identify your bag (on a computer screen!) you head off through security again and thence to your interview with the Immigration official. At least this time I visited the US I had completed an ESTA; when I travelled through the US on the way to Mexico 18 months ago I had forgotten this and got wrapped on the knuckles as a result (“non compliant” was what the official called me!). Once that was over with, you are effectively a domestic traveller like any other person travelling from Vancouver to San Francisco.

The journey from the airport to downtown was likewise uneventful on the BART (Bay Area Rapid Transit). It was just a shame that all this happened in the dark so I couldn’t see any of the city as I went along. My hotel is just round the corner from Union Square right in the heart of the shopping district and when I arrived the place was buzzing as befits 9pm on a Saturday night! Very lively atmosphere, what little I saw of it on my 5 minute trek to the hotel. My room mate had already checked in and was fast asleep when I arrive (it was 5am UK time by the time I got to the hotel) so I dumped my gear, crawled into bed and slept too.

This morning we had an expensive breakfast in the hotel (won’t be doing that again as there are plenty of other, cheaper options available!) then headed off for a walk in the sunshine before meeting up with some colleagues for lunch. After that we headed to the Moscone Center to register and that about brings us up to date. The real business starts tomorrow; I have a talk to give at 1500 in session H13G “Digital Soil Mapping and Hydroecological Modeling I”. This is a rather general soil/hydrology modelling session and I was originally invited to give a talk in an R-focussed session, so I’ve had to retool my talk to suit. This does mean that the session won’t be too interesting to me scientifically and I may try to sneak out (or in rather as I am on next to last) to a session on carbon isotopes. Whether to risk the session under-running or not?

Off to meet up with some people now and sample a beer or two before bedtime. I’ll be back tomorrow, Wi-Fi permitting with a round-up of Day 1 of the AGU 2011 Fall Meeting.

Image via Wikipedia

In previous posts I discussed how to generate a single permutation from a fully-randomised or restricted permutation design using shuffle(). Here I want to briefly mention the shuffleSet() function and illustrate it’s usage.

Every time you call shuffle() it has to interpret the control list to identify the type of permutation required. Whilst the overhead of this interpretation is not too high, there is no reason that it need be incurred just to generate a set of permutations. This is where shuffleSet() comes in. It works exactly like shuffle() taking the number of observations and a control object but in addition it takes an extra argument nset which is the number of permutations required for the set.

> require(permute) ## load the package
> args(shuffleSet)
function (n, nset = 1, control = permControl())
NULL

To generate 10 random permutations of ten observations you would use

> set.seed(2)
> shuffleSet(10, 10)
      [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
 [1,]    2    7    5   10    6    8    1    3    4     9
 [2,]    6    3    7    2    9    5    4    1   10     8
 [3,]    7    4   10    2    3    6    1    8    5     9
 [4,]    1    2    7    8    4    6    5   10    9     3
 [5,]   10    3    1    2    6    4    5    7    9     8
 [6,]    1   10    6    7    2    5    4    3    8     9
 [7,]    8   10    6    2    9    3    7    4    1     5
 [8,]    3   10    1    2    7    4    6    9    8     5
 [9,]    4    7    1    3    2    5   10    8    6     9
[10,]   10    4    9    8    3    1    2    5    6     7

If those 10 observations were collected as a time series and we wanted 10 restricted permutations you would use

> set.seed(2)
> shuffleSet(10, 10, control = permControl(within = Within(type = "series")))
      [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
 [1,]    3    4    5    6    7    8    9   10    1     2
 [2,]    9   10    1    2    3    4    5    6    7     8
 [3,]    7    8    9   10    1    2    3    4    5     6
 [4,]    3    4    5    6    7    8    9   10    1     2
 [5,]    1    2    3    4    5    6    7    8    9    10
 [6,]    1    2    3    4    5    6    7    8    9    10
 [7,]    3    4    5    6    7    8    9   10    1     2
 [8,]   10    1    2    3    4    5    6    7    8     9
 [9,]    6    7    8    9   10    1    2    3    4     5
[10,]    7    8    9   10    1    2    3    4    5     6

From the above set of permutations, the cyclic shifts employed in the "series" permutation type are clear. One problem with the set we just produced is that the same permutation was returned more than once. In fact, there were only six unique permutations in the set requested. This is due to there only being 10 possible permutations of the numbers 1, 2, …, 10 if we allow cyclic shifts in a single direction

> numPerms(10, control = permControl(within = Within(type = "series")))
[1] 10

shuffle() and shuffleSet() know nothing of these limits, but there are functions in the permute package that can tell you the number of possible permutations (numPerms()) and generate the entire set of permutations for a stated design (allPerms()). I’ll take a look at allPerms() in a future posting.

I return now to the Golden Jackal mandible length example I used in an earlier post but update the example to make use of shuffleSet() instead of shuffle(). I will just show the code and output for the permutation test, refer to the previous post for details:

> data(jackal) ## load the data
> ## function to compute the difference of means
> meanDif <- function(x, grp) {
+     mean(x[grp == "Male"]) - mean(x[grp == "Female"])
+ }
> N <- nrow(jackal)
> ## make this reproducible and like previous post
> set.seed(42)
> ## generate the set of 4999 random permutations
> pSet  <- shuffleSet(N, 4999)
> ## iterate over the set
> Djackal <- apply(pSet, 1, function(i, data) with(data, meanDif(Length, Sex[i])),
+                  data = jackal)
> ## add the observed difference of mean mandible length
> Djackal <- c(Djackal, with(jackal, meanDif(Length, Sex)))
## No. obs equal to or greater than observed
> (Dbig <- sum(Djackal >= Djackal[5000]))
[1] 12
> ## permutational p-value
> Dbig/length(Djackal)
[1] 0.0024

The last two lines of R code compute the number of observations in the Null distribution with differences in mean mandible length as great or greater than the observed difference, and the resulting permutation p-value. These are the same as those computed in the previous post.

Generating entire sets of permutations is useful for several reasons. One recent example that we came across is with the new parallel processing capabilities in the forthcoming version of R. We are able to generate a set of permutations and then distribute the process of the permutation test over a number of CPUs or worker threads, each dealing with a subset of the permutations we generated. This can greatly reduce the compute time needed for the permutation test, especially where the objective function is computationally complex, but allows us to not worry about controlling the random number generator in each separate process — this is all done within the main function and only the relevant subset of permutations is passed to each worker process. An additional reason for generating a set of permutation to work with rather than individual permutations is that it is easy to switch between using a set of randomly generated permutations or the set of all possible permutations where that set is not overly large. allPerms() returns the set of permutations in the same way that shuffleSet() does, so we can simplify our code if we write the test to iterate over a set of permutations.

The full script for the Golden Jackal permutation test is shown below:

data(jackal) ## load the data
## function to compute the difference of means
meanDif <- function(x, grp) {
    mean(x[grp == "Male"]) - mean(x[grp == "Female"])
}
N <- nrow(jackal)
## make this reproducible and like previous post
set.seed(42)
## generate the set of 4999 random permutations
pSet  <- shuffleSet(N, 4999)
## iterate over the set
Djackal <- apply(pSet, 1, function(i, data) with(data, meanDif(Length, Sex[i])),
                 data = jackal)
## add the observed difference of mean mandible length
Djackal <- c(Djackal, with(jackal, meanDif(Length, Sex)))
## No. obs equal to or greater than observed
(Dbig <- sum(Djackal >= Djackal[5000]))
## permutational p-value
Dbig/length(Djackal)

shuffle() has two arguments: i) n, the number of observations in the data set to be permuted, and ii) control, a list that defines the permutation design describing how the samples are permuted.

R> args(shuffle)
function (n, control = permControl())
NULL

control is a list, and for complex permutation designs. As a result, several convenience functions are provided that make it easier to specify the design you want. The main convenience function is permControl() which if passed no arguments populates an appropriate control object with defaults that result in free permutation of observations.

> str(permControl())
List of 10
 $ strata     : NULL
 $ nperm      : num 199
 $ complete   : logi FALSE
 $ within     :List of 5
  ..$ type    : chr "free"
  ..$ constant: logi FALSE
  ..$ mirror  : logi FALSE
  ..$ ncol    : NULL
  ..$ nrow    : NULL
 $ blocks     :List of 4
  ..$ type  : chr "none"
  ..$ mirror: logi FALSE
  ..$ ncol  : NULL
  ..$ nrow  : NULL
 $ maxperm    : num 9999
 $ minperm    : num 99
 $ all.perms  : NULL
 $ observed   : logi FALSE
 $ name.strata: chr "NULL"
 - attr(*, "class")= chr "permControl"

Several types of permutation can be produced by functions in permute:

• Free permutation of objects, which we saw in the earlier post
• Time series or line transect designs, where the temporal or spatial ordering is preserved
• Spatial grid designs, where the spatial ordering is preserved in both coordinate directions
• Permutation of blocks or groups of samples

The first three of these can be nested within the levels of a factor or to the levels of that factor, or to both. Such flexibility allows the analysis of split-plot designs using permutation tests. permControl() is used to set up the design from which shuffle() will draw a permutation. permControl() has two main arguments that specify how samples are permuted within blocks of samples or at the block level itself. These are within and blocks. Two convenience functions, Within() and Blocks() can be used to set the various options for permutation. For example, to permute the observations 1:10 assuming a time series design for the entire set of observations, the following control object would be used

> set.seed(4)
> x <- 1:10
> CTRL <- permControl(within = Within(type = "series"))
> perm <- shuffle(10, control = CTRL)
> perm
 [1]  7  8  9 10  1  2  3  4  5  6
> x[perm]
 [1]  7  8  9 10  1  2  3  4  5  6

A more complex design, with three blocks, and a 3 by 3 spatial grid arrangement within each block can be created as follows

> set.seed(4)
> block <- gl(3, 9)
> CTRL <- permControl(strata = block,
+                     within = Within(type = "grid", ncol = 3, nrow = 3))
> perm <- shuffle(length(block), control = CTRL)
> perm
 [1]  6  4  5  9  7  8  3  1  2 14 15 13 17 18 16 11 12 10 22 23
[21] 24 25 26 27 19 20 21

> ## Original
> lapply(split(1:27, block), matrix, ncol = 3)
$`1`
     [,1] [,2] [,3]
[1,]    1    4    7
[2,]    2    5    8
[3,]    3    6    9

$`2`
     [,1] [,2] [,3]
[1,]   10   13   16
[2,]   11   14   17
[3,]   12   15   18

$`3`
     [,1] [,2] [,3]
[1,]   19   22   25
[2,]   20   23   26
[3,]   21   24   27

> ## Shuffled
> lapply(split(perm, block), matrix, ncol = 3)
$`1`
     [,1] [,2] [,3]
[1,]    6    9    3
[2,]    4    7    1
[3,]    5    8    2

$`2`
     [,1] [,2] [,3]
[1,]   14   17   11
[2,]   15   18   12
[3,]   13   16   10

$`3`
     [,1] [,2] [,3]
[1,]   22   25   19
[2,]   23   26   20
[3,]   24   27   21

To have the same permutation within each level of block, use the constant argument of the Within() function, setting it to TRUE

> set.seed(4)
> CTRL <- permControl(strata = block,
+                     within = Within(type = "grid", ncol = 3, nrow = 3,
+                                     constant = TRUE))
> perm2 <- shuffle(length(block), control = CTRL)
> lapply(split(perm2, block), matrix, ncol = 3)
$`1`
     [,1] [,2] [,3]
[1,]    6    9    3
[2,]    4    7    1
[3,]    5    8    2

$`2`
     [,1] [,2] [,3]
[1,]   15   18   12
[2,]   13   16   10
[3,]   14   17   11

$`3`
     [,1] [,2] [,3]
[1,]   24   27   21
[2,]   22   25   19
[3,]   23   26   20

As you can see, at the moment, I make some assumptions about the ordering of samples within each spatial/temporal structure. The samples do not have the be arranged in strata order, but within the levels of the grouping variable the observations must be in the right order. For spatial grids, this means in column-major order—just as in the way R fills matrices by columns. In a future release, I hope to relax some of these assumptions to make it easier to apply permutations to the data to hand.

In the next post in this series, I’ll take a look at generating sets of permutations using the shuffleSet() function.

Instead, permutation tests are used to form an appropriate Null distribution for a test statistic against which we can evaluate the observed value of that statistic. In constrained ordination the test statistic is usually in the form of a pseudo F statistic, although one advantage of the permutation approach is that any suitable statistic can be used in the test. If the observed value of the test statistic is unusual in the context of the Null distribution derived by permuting the data, say in the top 5 or 1 percent, we conclude that the observed relationship is unlikely to have arisen by chance and is therefore significant.

Simply shuffling the rows of the input data, a process also known as a randomisation test, is sufficient where the observations in the data are independent. If the observations are independent, under the Null model any row from the species data can be matched with any row from the predictor data, which justifies the use of randomisation as the basis for testing the significance of the model.

In many cases, however, ecological data are not independent, having been collected sequentially in space (a transect) or time (time series). Designed experiments are commonplace, with replicates located within plots that are experimentally manipulated. Observations might also have been repeatedly made from within a number of sampling locations or plots, or from a spatial grid over the region of interest.

The vegan package has long had permutation tests, based on the function permuted.index(). This function allowed for simple randomisation, or randomisation within groups, defined by a factor supplied as argument strata.

CANOCO has long had support for these restricted permutations methods. I have been working sporadically for several years developing code initially in vegan and then under the banner of the permute package to implement similar functionality.

In a series of posts over the coming weeks I will explain what permute can do and illustrate how to use the package. Over time, myself and the other vegan developers will start interfacing existing functions in the package that use permutations with the permute package so that gradually the scope of permutation tests in vegan will improve.

To get you started though, here is a quick, simple example of doing a randomisation test using permute…

We consider a small data set of mandible length measurements on specimens of the golden jackal (Canis aureus) from the British Museum of Natural History, London, UK. These data were collected as part of a study comparing prehistoric and modern canids (Higham et al. 1980), and were previously analysed by Manly (2007). There are ten measurements of mandible length on both male and female specimens. The data are available in the jackal data frame supplied with permute. Load the package and the data set

> require(permute)
> data(jackal)
> jackal

The interest is in whether there is a difference in the mean mandible length between male and female golden jackals. The null hypothesis is that there is zero difference in mandible length between the two sexes or that females have larger mandibles. The alternative hypothesis is that males have larger mandibles. The usual statistical test of this hypothesis is a one-sided t test, which can be applied using t.test()

> jack.t <- t.test(Length ~ Sex, data = jackal, var.equal = TRUE,
+    alternative = "greater")
> jack.t

	Two Sample t-test

data:  Length by Sex 
t = 3.4843, df = 18, p-value = 0.001324
alternative hypothesis: true difference in means is greater than 0 
95 percent confidence interval:
 2.411156      Inf 
sample estimates:
  mean in group Male mean in group Female 
               113.4                108.6

A permutation-based test can be used to test the same Null hypothesis, but without some of the assumptions of the t test, most importantly the assumption that the data are a random sample from the population of golden jackals. With a permutation test, we are free to choose any suitable test statistic. We could use the t statistic, but the difference in means of the Female and Male groups will suffice. To implement this we build a function that will compute the difference of means for the Male and Female groups:

meanDif <- function(x, grp) {
    mean(x[grp == "Male"]) - mean(x[grp == "Female"])
}

This function can be used in a for() loop to generate the permutation distribution of the test statistic under the Null hypothesis. We will perform 4999 random permutations, so we allocate a vector of length 5000 to hold the resulting difference of means. Under the Null hypothesis, the observed difference of means is just one of the possible values so we count it as part of the Null distribution (hence the length of 5000). In the code chunk below, Djackal will contain the 5000 differences of means for the Null distribution, N holds the number of observations in the jackal data set. Then we seed the pseudo-random number generator to get reproducible results and initiate a loop to generate the Null distribution, of which more in a minute. The last line adds the observed difference of means to the Null distribution

> Djackal <- numeric(length = 5000)
> N <- nrow(jackal)
> set.seed(42)
> for (i in seq_len(length(Djackal) - 1)) {
+     perm <- shuffle(N)
+     Djackal[i] <- with(jackal, meanDif(Length, Sex[perm]))
+ }
> Djackal[5000] <- with(jackal, meanDif(Length, Sex))

The loop runs over the values {1,…4999} and generates a randomisation of the N rows in the jackal data frame using the shuffle() function. The second line in the loop computes the difference in mean mandible length for the permuted data. shuffle() is one of the key functions available in permute. When called, as here, with a single argument (the number of observations) it returns a random permutation of the integers {1, …, N}. In fact, it works very much as a wrapper for the base R function sample():

> set.seed(2)
> (r1 <- shuffle(10))
 [1]  2  7  5 10  6  8  1  3  4  9
> set.seed(2)
> (r2 <- sample(1:10, 10, replace = FALSE))
 [1]  2  7  5 10  6  8  1  3  4  9
> all.equal(r1, r2)
[1] TRUE

> hist(Djackal, main = "",
+      xlab = expression("Mean difference (Male - Female) in mm"))
> rug(Djackal[5000], col = "red", lwd = 2)

Permutation Null distribution for difference of mean mandible length. The red tick mark indicates the observed difference in mean mandible lengths

The observed difference of mean mandible length is located in the extreme right tail of the Null distribution. The number of permuted difference of mean lengths that are equal to or greater than the observed difference is 12, yielding a permutational p value of 0.0024

> (Dbig <- sum(Djackal >= Djackal[5000]))
[1] 12
> Dbig/length(Djackal)
[1] 0.0024

I hope that has whet your appetite? In future posts I’ll explain more about how permute works and explain how to use it to generate restricted permutations.

References:
Higham C, Kijngam A, Manly B (1980). An analysis of prehistoric canid remains from Thailand. Journal of Archaeological Science, 7, 149–165.
Manly B (2007). Randomization, bootstrap and Monte Carlo methods in biology. 3rd edition. Chapman & Hall/CRC, Boca Raton.

FOO <- data.frame(x = 1:10, y = 1:10)
rownames(FOO) <- LETTERS[1:10]
class(FOO) <- "foo"

A simplified plot() method to plot the x and y components of our object, displaying the data as points or text labels might be:

plot.foo <- function(X, type = c("points","text"), ...) {
    x <- X$x
    y <- X$y
    type <- match.arg(type)
    plot(x, type = "n", ...)
    if(type == "points") {
        points(x, y, ...)
    } else {
        text(x, y, labels = rownames(x), ...)
    }
    invisible(x)
}

Note that we are passing ... on to each of plot(), points(), and text() so our method is very simple. However, if we try to suppress the drawing of axes using the axes argument of plot.default(), our method will generate errors

> plot(FOO, axes = FALSE)
Warning message:
In plot.xy(xy.coords(x, y), type = type, ...) :
  "axes" is not a graphical parameter

Turning warnings into errors, we see that the call to points() is where the warning originates (actually in plot.xy(), frame 5, but points() is the offending code in our method)

> options(warn = 2) ## turn warnings to errors
> 
> plot(FOO, axes = FALSE)
Error in plot.xy(xy.coords(x, y), type = type, ...) : 
  (converted from warning) "axes" is not a graphical parameter
> ## look at the call stack
> traceback()
9: doWithOneRestart(return(expr), restart)
8: withOneRestart(expr, restarts[[1L]])
7: withRestarts({
       .Internal(.signalCondition(simpleWarning(msg, call), msg, 
           call))
       .Internal(.dfltWarn(msg, call))
   }, muffleWarning = function() NULL)
6: .signalSimpleWarning("\"axes\" is not a graphical parameter", 
       quote(plot.xy(xy.coords(x, y), type = type, ...)))
5: plot.xy(xy.coords(x, y), type = type, ...)
4: points.default(x, y, ...)
3: points(x, y, ...)
2: plot.foo(FOO, axes = FALSE)
1: plot(FOO, axes = FALSE)

The warning results from our function passing axes = FALSE on to the lower-level plotting functions.

An obvious solution is to process ... and strip out any offending non-graphical parameters and then arrange for the calls to use the stripped out .... Doing this is possible, but is very complicated. There is an alternative, simpler solution that is used in several base R functions and suggested to me by Brian Ripley (when I asked about doing this on R-Help for a function in the vegan package). The trick is to have a local, in-line wrapper around points() of the following form:

lPoints <- function(..., log, axes, frame.plot, panel.first, panel.last) { 
    points(...)
}

Here we list all the arguments of plot.default() we don’t want passed on to the low-level plotting calls, but importantly, they are listed after .... The only code in the body of the local function is a call to the low-level graphics function we want to use. Importantly, of the arguments taken by lPoints() only ... is passed on to the graphics function it wraps. Because the arguments from plot.default() are named and come after ... in the definition of lPoints(), any arguments passed to lPoints() that fully match the named arguments are automatically stripped from the ... that is passed on to the wrapped function.

Using this trick, we can now write our plot.foo() method like this:

plot.foo <- function(X, type = c("points","text"), ...) {
    lPoints <- function(..., log, axes, frame.plot,
                        panel.first, panel.last) points(...)
    lText <- function(..., log, axes, frame.plot,
                      panel.first, panel.last) text(...)
    x <- X$x
    y <- X$y
    type <- match.arg(type)
    plot(x, type = "n", ...)
    if(type == "points") {
        lPoints(x, y, ...)
    } else {
        lText(x, y, labels = rownames(x), ...)
    }
    invisible(x)
}

Now we can pass arguments to both plot.default() and points() and text(), and the call that raised the warning earlier, now works without complaint:

> plot(FOO, axes = FALSE)
>

Remember to reset the warning level if you followed the code above

options(warn = 0)