[R] Error: vector memory exhausted (limit reached?)

[Rui Barradas]

Hello,

Here is your code, simplified and maybe corrected.
Some previous notes:

1. You don't need to attach()

2. You don't need to create categorydata<- as.data.frame(OrigData), 
read.csv already outputs a data.frame. In the code below, I use Origdata.

3. The file has many missing values, so I have cleaned the data. The 
code lines with the new pipe operator (R 4.1.0) give the dim of OrigData 
after cleansing and create the data.frame CleanData, with all complete 
rows, no missing values. There are a total of 968 rows with missing 
values so CleanData only has 840 rows.

4. Instead of having a very long regression formula, I use grep to get 
the column names starting with "B6_" and then a series of paste 
assembles the regression formula.



library(relaimpo)

# Read the data in. I've downloaded it to this directory
path <- "~/tmp"
filename <- "BB HTTF stacked B1, B3, B5, B6, C9 + FILTERED.csv"
filename <- file.path(path, filename)
OrigData <- read.csv(filename)

###### BRAND PERFORMANCE ADVOCACY (B5) ####

# Runs a standard linear regression.

resp <- "B5"
regr_names <- grep("B6_", names(OrigData), value = TRUE)
regr <- paste(regr_names, collapse = " + ")
fmla <- as.formula(paste(resp, regr, sep = " ~ "))

dim(OrigData)
#[1] 1808   63
OrigData[c(resp, regr_names)] |> na.omit() |> dim()
#[1] 840  28

CleanData <- OrigData[c(resp, regr_names)] |> na.omit()

linmod4 <- lm(fmla, data = OrigData, na.action = na.exclude)
linmod4_b <- lm(fmla, data = CleanData)

all(coef(linmod4) == coef(linmod4_b))
#[1] TRUE

# Runs Shapley Value Regression with all the
# coefficients set to sum a hundred

# Any of these first two equivalent forms
# take a very long time
f4_lmg <- calc.relimp(
   linmod4_b,
   type = "lmg",
   rela = TRUE
)
f4_lmg_b <- calc.relimp(
   formula = fmla,
   type = "lmg",
   rela = TRUE,
   data = CleanData
)

# These equivalent forms are quickly done
f4_firstlast <- calc.relimp(
   linmod4_b,
   type = c("first","last"),
   rela = TRUE
)
f4_firstlast_b <- calc.relimp(
   formula = fmla,
   type = c("first","last"),
   rela = TRUE,
   data = CleanData
)

Coefficient4 <- f4_lmg$lmg
Rsq4 <- f4_firstlast$R2
Proportion4 <- Coefficient4 * Rsq4



Hope this helps,

Rui Barradas



Às 13:36 de 24/11/21, Olivia Keefer escreveu:
> Apologies. First time posting and new to R so a lot of learning. Hope my attachments are helpful now.
> 
> 
> #This loads the required package. Always select UK Bristol as a CRAN MIRROR / LOCATION
> #1. Highlight the below code and run (3rd icon or right click)
> 
> require(relaimpo)
> install.packages('relaimpo',dep=TRUE)
> install.packages('iterators')
> install.packages('foreach')
> 
> #2. Change directory to where all files are and change back slash to forward slash
> 
> setwd('/Users/okeefer/Documents/Regressions')
> 
> 
> 
> #Once final SPSS file is created save it as a CSV in same location.
> #4. Below code opens the original file with data at respondent level. Update the name of data file only.
> 
> OrigData = read.csv("BB HTTF stacked B1, B3, B5, B6, C9 + FILTERED.csv")
> 
> #5. <Run below code to bottom do not need to change anything
> #Shows all the variables names of the dataset
> 
> names(OrigData)
> 
> #Shows all the dimensions and class of the dataset
> 
> dim(OrigData)
> class(OrigData)
> 
> #Creates a data frame
> 
> categorydata<- as.data.frame(OrigData)
> 
> #Shows all the variables & class from the newly created variable
> 
> names(categorydata)
> class(categorydata)
> 
> #Makes the object accessible
> 
> attach(categorydata)
> 
> #>
> 
> 
> ######BRAND PERFORMANCE ADVOCACY (B5)####
> 
> #Runs a standard linear regression. Update predictor and outcome variables. The variable names only.
> #First variable is Dependent variable. Ensure a + sign is between all independent variables. Ensure there is a space before and after +
> 
> linmod4 <- lm( B5 ~ B6_1 + B6_2 + B6_3 + B6_4 + B6_5 + B6_6 + B6_7 + B6_8 + B6_9 + B6_10 + B6_11 + B6_12 + B6_13 + B6_14 + B6_15 + B6_16 + B6_17 + B6_18 + B6_19 + B6_20 + B6_21 + B6_22 + B6_23 + B6_24 + B6_25 + B6_26 + B6_27 , data= categorydata)
> 
> #Runs Shapley Value Regression with all the coefficients set to sum a hundred
> 
> f4 <-calc.relimp(linmod4,type = c("lmg","first","last"), rela = TRUE)
> 
> Coefficient4 = f4$lmg
> Rsq4 = f4$R2
> Proportion4 = Coefficient4 * Rsq4
> 
> ### Display CoEfficients
> 
> Coefficient4
> 
> ### Display R-Sq
> 
> Rsq4
> 
> ### Proportion of Model Explained by variable. Change file name to reflect project
> Proportion4
> 
> Results4 = cbind(Coefficient4,Proportion4,Rsq4)
> write.csv(Results4, file = "bb_HTTF_performance_advocacy.csv")
> 
> 
> Olivia Keefer
> Insights Analyst
> she/her/hers
>   
> Monigle
> 575 8th Avenue
> Suite 1716
> New York, NY 10018
> M 740.701.2163
> okeefer using monigle.com
> www.monigle.com <http://www.monigle.com/>
> linkedin <http://www.linkedin.com/company/monigle/>| twitter <https://twitter.com/Monigle>| monigle blog <http://www.monigle.com/blog/>
>   
> 
>   
> 
> On 11/24/21, 2:59 AM, "Rui Barradas" <ruipbarradas using sapo.pt> wrote:
> 
>      Hello,
> 
>      You ask a question on a certain regression that exhausts vector memory
>      but don't post the regression(s) code (and data, btw).
>      And load package relaimpo before installing it.
> 
>      Can you please read the posting guide linked to at the bottom of this
>      and every R-Help mail? As is there's nothing to answer to.
> 
>      Hope this helps,
> 
>      Rui Barradas
> 
> 
>      Às 20:26 de 23/11/21, Olivia Keefer escreveu:
>      > Hello!
>      >
>      > My colleague and I have continually run into this error and would like to better understand why and how to remedy. We have been able to run other regressions, but when we go to run a certain set of variables, we both are getting this message each time we try.
>      >
>      > Any insight would be helpful as to 1) why and 2) how to remedy.
>      >
>      > Attaching our package as well here if it is helpful:
>      >
>      >
>      >
>      > #This loads the required package. Always select UK Bristol as a CRAN MIRROR / LOCATION
>      >
>      > #1. Highlight the below code and run (3rd icon or right click)
>      >
>      >
>      >
>      > require(relaimpo)
>      >
>      > install.packages('relaimpo',dep=TRUE)
>      >
>      > install.packages('iterators')
>      >
>      > install.packages('foreach')
>      >
>      >
>      > Thanks!
>      >
>      > Olivia Keefer
>      > Insights Analyst
>      > she/her/hers
>      >
>      > Monigle
>      > 575 8th Avenue
>      > Suite 1716
>      > New York, NY 10018
>      > M 740.701.2163
>      > okeefer using monigle.com<mailto:okeefer using monigle.com>
>      > www.monigle.com<http://www.monigle.com/>
>      > linkedin<http://www.linkedin.com/company/monigle/>| twitter<https://twitter.com/Monigle>| monigle blog<http://www.monigle.com/blog/>
>      >
>      >
>      >
>      >       [[alternative HTML version deleted]]
>      >
>      > ______________________________________________
>      > R-help using r-project.org mailing list -- To UNSUBSCRIBE and more, see
>      > https://stat.ethz.ch/mailman/listinfo/r-help
>      > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>      > and provide commented, minimal, self-contained, reproducible code.
>      >
>      NOTICE: This email originated outside of Monigle Associates. Please be diligent about verifying the sender and validity of this message. Report this message if necessary.
>