Greetings,

I am a landscape geneticist, and have been attempting to develop some methods to incorporate ordination for individual-based analyses. Unfortunately, individual-based analyses necessitate the use of distances because hypotheses involve quantifying the relationship between some measure of genetic differentiation (often pair-wise as well) and landscape resistance (calculated using Least cost paths for example). The natural choice has always been partial mantel tests, but lots of recent studies have questioned their validity and accuracy. 

My methods have involved simulating a single population where movement (and gene flow) are dictated by either geographic distance alone or landscape+geographic distance. I am interested in being able to control for the effects of geographic distance to isolate the effect of the landscape. The correlation between landscape resistance (cost distance) and geographic distance is always very strong, but for some reason, regardless of the test (cca, rda, and capscale), I always get significant results even when I "controlled" for geographic distance. I think I may need a better method to control for geographic distance, but have come up empty so far. 

My apologies for the big introduction, but I figure it would help in understanding. This is the method I have employed to control for geographic distance:

In this example: 400 individuals were simulated on a homogenous landscape, so after controlling for Euclidean distance, cost distance is not expected to be significant. 

I calculated cost distance (accumulated cost of traveling through a landscape along a Euclidean line) between all individuals in ArcGIS. In a homogeneous landscape, the Euclidean and cost distances were almost identical (Mantel r= 0.9998). I transformed both the Euclidean distances and cost distances using a PCoA: 

#pair-wise matrix of cost distances between 400 individuals on a homogeneous landscape#
Cost1<-read.table(file="Cost1.txt", header=F)
#PCoA of pair-wise matrix, I limited the dimensions to 2 because I controlled gene flow to occur in 2 dimensions#
CostPCO<-cmdscale(Cost1, k=2) 

I performed the same for Euclidean distance. 

Next, I extracted the scores for each of PCoAs and performed a procrustes rotation to maximize the similarity between the two matrices. Finally, I extracted the residuals from the procrustes rotation as that (in my mind) should account for the tight correlation between the two distance matrices. 

#vectors of PCoA scores for Cost distance#
Cost.scores<-scores(CostPCO)
#vectors of PCoA scores for Euclidean distance#
ED.scores<-scores(EDPCO)
#procrustes rotation#
C1.pro<-procrustes(Cost.scores, ED.scores, symmetric=TRUE) 
#Extract residuals from procrustes rotation#
C1.res<-residuals(C1.pro)

The residuals were quite small, as expected, but things went haywire when I performed a cca, rda, and dbRDA on allele frequencies (cca and rda) or genetic distances (dbRDA). Almost all of my constrained ordination tests were significant, which does not make any sense. Here's an example code for the constrained ordination (note: there were 100 independently simulated populations). 
#dbRDA#
GD<-read.table(file="GD1.txt", header=F)
db<-capscale(GD~C1.res)
db.results<-anova.cca(db, by="axis")

I am completely stuck, over 80% of my dbRDAs were significant, which translates to an 80% type I error. CCA and RDA on allele frequencies have a 50% type I error rate. I am at a loss to why this is occurring. Any input would be wonderful, I think ordination has great potential in individual-based landscape genetics, but I cannot think of a way to get around this problem. Many many many thanks in advance. 

~Liz Kierepka
	[[alternative HTML version deleted]]

