Explain the difference between multiple regression and multivariate regression, with minimal use of symbols/math

  • Are multiple and multivariate regression really different? What is a variate anyways?

  • chl

    chl Correct answer

    10 years ago

    Very quickly, I would say: 'multiple' applies to the number of predictors that enter the model (or equivalently the design matrix) with a single outcome (Y response), while 'multivariate' refers to a matrix of response vectors. Cannot remember the author who starts its introductory section on multivariate modeling with that consideration, but I think it is Brian Everitt in his textbook An R and S-Plus Companion to Multivariate Analysis. For a thorough discussion about this, I would suggest to look at his latest book, Multivariable Modeling and Multivariate Analysis for the Behavioral Sciences.

    For 'variate', I would say this is a common way to refer to any random variable that follows a known or hypothesized distribution, e.g. we speak of gaussian variates $X_i$ as a series of observations drawn from a normal distribution (with parameters $\mu$ and $\sigma^2$). In probabilistic terms, we said that these are some random realizations of X, with mathematical expectation $\mu$, and about 95% of them are expected to lie on the range $[\mu-2\sigma;\mu+2\sigma]$ .

    Even https://www.coursera.org/learn/machine-learning/home/week/2 uses the term multivariate regression instead of multiple regression…

    I think the same confusion arises with people using the term GLM for General Linear Model (e.g., in neuroimaging studies) vs. Generalised Linear Model. I have seen many instances of "multivariate logistic regression" where there's only one outcome, and I don't think this matters so much as long as the term is clearly defined by the author.

License under CC-BY-SA with attribution


Content dated before 6/26/2020 9:53 AM