| Title: | Supervised Tensor Decomposition with Side Information |
|---|---|
| Description: | Implement the alternating algorithm for supervised tensor decomposition with interactive side information. Details can be found in the publication Hu, Jiaxin, Chanwoo Lee, and Miaoyan Wang. "Generalized Tensor Decomposition with features on multiple modes." Journal of Computational and Graphical Statistics, Vol. 31, No. 1, 204-218, 2022 <doi:10.1080/10618600.2021.1978471>. |
| Authors: | Jiaxin Hu [aut, cre, cph], Chen Zhang [aut, cph], Chanwoo Lee [aut, cph], Miaoyan Wang [aut, cph] |
| Maintainer: | Jiaxin Hu <[email protected]> |
| License: | GPL (>= 2) |
| Version: | 5.1 |
| Built: | 2025-03-12 03:12:57 UTC |
| Source: | https://github.com/cran/tensorregress |
Create a Tensor-class object from an array, matrix, or vector.
as.tensor(x, drop = FALSE)as.tensor(x, drop = FALSE)
x |
an instance of |
drop |
whether or not modes of 1 should be dropped |
a Tensor-class object
#From vector vec <- runif(100); vecT <- as.tensor(vec); vecT #From matrix mat <- matrix(runif(1000),nrow=100,ncol=10) matT <- as.tensor(mat); matT #From array indices <- c(10,20,30,40) arr <- array(runif(prod(indices)), dim = indices) arrT <- as.tensor(arr); arrT#From vector vec <- runif(100); vecT <- as.tensor(vec); vecT #From matrix mat <- matrix(runif(1000),nrow=100,ncol=10) matT <- as.tensor(mat); matT #From array indices <- c(10,20,30,40) arr <- array(runif(prod(indices)), dim = indices) arrT <- as.tensor(arr); arrT
Return the vector of modes from a tensor
## S4 method for signature 'Tensor' dim(x)## S4 method for signature 'Tensor' dim(x)
x |
the Tensor instance |
dim(x)
an integer vector of the modes associated with x
tnsr <- rand_tensor() dim(tnsr)tnsr <- rand_tensor() dim(tnsr)
General folding of a matrix into a Tensor. This is designed to be the inverse function to unfold-methods, with the same ordering of the indices. This amounts to following: if we were to unfold a Tensor using a set of row_idx and col_idx, then we can fold the resulting matrix back into the original Tensor using the same row_idx and col_idx.
fold(mat, row_idx = NULL, col_idx = NULL, modes = NULL)fold(mat, row_idx = NULL, col_idx = NULL, modes = NULL)
mat |
matrix to be folded into a Tensor |
row_idx |
the indices of the modes that are mapped onto the row space |
col_idx |
the indices of the modes that are mapped onto the column space |
modes |
the modes of the output Tensor |
This function uses aperm as the primary workhorse.
Tensor object with modes given by modes
T. Kolda, B. Bader, "Tensor decomposition and applications". SIAM Applied Mathematics and Applications 2009, Vol. 51, No. 3 (September 2009), pp. 455-500. URL: https://www.jstor.org/stable/25662308.
tnsr <- new('Tensor',3L,c(3L,4L,5L),data=runif(60)) matT3<-unfold(tnsr,row_idx=2,col_idx=c(3,1)) identical(fold(matT3,row_idx=2,col_idx=c(3,1),modes=c(3,4,5)),tnsr)tnsr <- new('Tensor',3L,c(3L,4L,5L),data=runif(60)) matT3<-unfold(tnsr,row_idx=2,col_idx=c(3,1)) identical(fold(matT3,row_idx=2,col_idx=c(3,1),modes=c(3,4,5)),tnsr)
The array "tensor" is a 68 × 68 × 136 binary tensor consisting of structural connectivity patterns among 68 brain regions for 136 individuals from Human Connectome Project (HCP). All the individual images were preprocessed following a standard pipeline (Zhang et al., 2018), and the brain was parcellated to 68 regions-of-interest following the Desikan atlas (Desikan et al., 2006). The tensor entries encode the presence or absence of fiber connections between those 68 brain regions for each of the 136 individuals. The data frame "attr" is a 136 × 573 matrix consisting of 573 personal features for 136 individuals. The full list of covariates can be found at: https://wiki.humanconnectome.org/display/PublicData/
data(HCP)data(HCP)
A list. Includes a 68-68-136 binary array named "tensor" and a 136-573 data frame named "attr".
Higher-order SVD of a K-Tensor. Write the K-Tensor as a (m-mode) product of a core Tensor (possibly smaller modes) and K orthogonal factor matrices. Truncations can be specified via ranks (making them smaller than the original modes of the K-Tensor will result in a truncation). For the mathematical details on HOSVD, consult Lathauwer et. al. (2000).
hosvd(tnsr, ranks = NULL)hosvd(tnsr, ranks = NULL)
tnsr |
Tensor with K modes |
ranks |
a vector of desired modes in the output core tensor, default is |
A progress bar is included to help monitor operations on large tensors.
a list containing the following:
Zcore tensor with modes speficied by ranks
Ua list of orthogonal matrices, one for each mode
estestimate of tnsr after compression
fnorm_residthe Frobenius norm of the error fnorm(est-tnsr) - if there was no truncation, then this is on the order of mach_eps * fnorm.
The length of ranks must match tnsr@num_modes.
L. Lathauwer, B.Moor, J. Vandewalle, "A multilinear singular value decomposition". Journal of Matrix Analysis and Applications 2000, Vol. 21, No. 4, pp. 1253–1278.
tnsr <- rand_tensor(c(6,7,8)) hosvdD <-hosvd(tnsr) hosvdD$fnorm_resid hosvdD2 <-hosvd(tnsr,ranks=c(3,3,4)) hosvdD2$fnorm_residtnsr <- rand_tensor(c(6,7,8)) hosvdD <-hosvd(tnsr) hosvdD$fnorm_resid hosvdD2 <-hosvd(tnsr,ranks=c(3,3,4)) hosvdD2$fnorm_resid
Returns the Kronecker product from a list of matrices or vectors. Commonly used for n-mode products and various Tensor decompositions.
kronecker_list(L)kronecker_list(L)
L |
list of matrices or vectors |
matrix that is the Kronecker product
smalllizt <- list('mat1' = matrix(runif(12),ncol=4), 'mat2' = matrix(runif(12),ncol=4), 'mat3' = matrix(runif(12),ncol=4)) dim(kronecker_list(smalllizt))smalllizt <- list('mat1' = matrix(runif(12),ncol=4), 'mat2' = matrix(runif(12),ncol=4), 'mat3' = matrix(runif(12),ncol=4)) dim(kronecker_list(smalllizt))
The array "R" is a 14 × 14 × 56 binary tensor consisting of 56 political relations of 14 countries between 1950 and 1965. The tensor entry indicates the presence or absence of a political action, such as “treaties”, “sends tourists to”, between the nations. Please set the diagonal elements Y(i,i,k) = 0 in the analysis. The matrix "cov" is a 14 × 6 matrix describing a few important country attributes, e.g. whether a nation is actively involved in medicine NGO, law NGO, or belongs to a catholic nation, etc.
data(nations)data(nations)
A list. Includes a 14-14-56 binary array named "R" and a 14-6 matrix named "cov".
Generate a Tensor with specified modes with iid normal(0,1) entries.
rand_tensor(modes = c(3, 4, 5), drop = FALSE)rand_tensor(modes = c(3, 4, 5), drop = FALSE)
modes |
the modes of the output Tensor |
drop |
whether or not modes equal to 1 should be dropped |
a Tensor object with modes given by modes
Default rand_tensor() generates a 3-Tensor with modes c(3,4,5).
rand_tensor() rand_tensor(c(4,4,4)) rand_tensor(c(10,2,1),TRUE)rand_tensor() rand_tensor(c(4,4,4)) rand_tensor(c(10,2,1),TRUE)
Estimate the Tucker rank of tensor decomposition based on BIC criterion. The choice of BIC aims to balance between the goodness-of-fit for the data and the degree of freedom in the population model.
sele_rank( tsr, X_covar1 = NULL, X_covar2 = NULL, X_covar3 = NULL, rank_range, niter = 10, cons = "non", lambda = 0.1, alpha = 1, solver = "CG", dist, initial = c("random", "QR_tucker") )sele_rank( tsr, X_covar1 = NULL, X_covar2 = NULL, X_covar3 = NULL, rank_range, niter = 10, cons = "non", lambda = 0.1, alpha = 1, solver = "CG", dist, initial = c("random", "QR_tucker") )
tsr |
response tensor with 3 modes |
X_covar1 |
side information on first mode |
X_covar2 |
side information on second mode |
X_covar3 |
side information on third mode |
rank_range |
a matrix containing rank candidates on each row |
niter |
max number of iterations if update does not convergence |
cons |
the constraint method, "non" for without constraint, "vanilla" for global scale down at each iteration, "penalty" for adding log-barrier penalty to object function. |
lambda |
penalty coefficient for "penalty" constraint |
alpha |
max norm constraint on linear predictor |
solver |
solver for solving object function when using "penalty" constraint, see "details" |
dist |
distribution of response tensor, see "details" |
initial |
initialization of the alternating optimiation, "random" for random initialization, "QR_tucker" for deterministic initialization using tucker decomposition |
For rank selection, recommend using non-constraint version.
Constraint penalty adds log-barrier regularizer to
general object function (negative log-likelihood). The main function uses solver in function "optim" to
solve the objective function. The "solver" passes to the argument "method" in function "optim".
dist specifies three distributions of response tensor: binary, poisson and normal distributions.
a list containing the following:
rank a vector with selected rank with minimal BIC
result a matrix containing rank candidate and its loglikelihood and BIC on each row
seed=24 dist='binary' data=sim_data(seed, whole_shape = c(20,20,20), core_shape=c(3,3,3),p=c(5,5,5),dist=dist, dup=5, signal=4) rank_range = rbind(c(3,3,3),c(3,3,2),c(3,2,2),c(2,2,2),c(3,2,3)) re = sele_rank(data$tsr[[1]],data$X_covar1,data$X_covar2,data$X_covar3, rank_range = rank_range,niter=10,cons = 'non',dist = dist,initial = "random")seed=24 dist='binary' data=sim_data(seed, whole_shape = c(20,20,20), core_shape=c(3,3,3),p=c(5,5,5),dist=dist, dup=5, signal=4) rank_range = rbind(c(3,3,3),c(3,3,2),c(3,2,2),c(2,2,2),c(3,2,3)) re = sele_rank(data$tsr[[1]],data$X_covar1,data$X_covar2,data$X_covar3, rank_range = rank_range,niter=10,cons = 'non',dist = dist,initial = "random")
Generate tensor data with multiple side information matrices under different simulation models, specifically for tensors with 3 modes
sim_data( seed = NA, whole_shape = c(20, 20, 20), core_shape = c(3, 3, 3), p = c(3, 3, 0), dist, dup, signal, block = rep(FALSE, 3), ortho = FALSE )sim_data( seed = NA, whole_shape = c(20, 20, 20), core_shape = c(3, 3, 3), p = c(3, 3, 0), dist, dup, signal, block = rep(FALSE, 3), ortho = FALSE )
seed |
a random seed for generating data |
whole_shape |
a vector containing dimension of the tensor |
core_shape |
a vector containing Tucker rank of the tensor decomposition |
p |
a vector containing numbers of side information on each mode, see "details" |
dist |
distribution of response tensor, see "details" |
dup |
number of simulated tensors from the same linear predictor |
signal |
a scalar controlling the max norm of the linear predictor |
block |
a vector containing boolean variables, see "details" |
ortho |
if "TRUE", generate side information matrices with orthogonal columns; if "FLASE" (default), generate side information matrices with gaussian entries |
By default non-positive entry in p indicates no covariate on the corresponding mode of the tensor.
dist specifies three distributions of response tensor: binary, poisson or normal distribution.
block specifies whether the coefficient factor matrix is a membership matrix, set to TRUE when utilizing the stochastic block model
a list containing the following:
tsr a list of simulated tensors, with the number of replicates specified by dup
X_covar1 a matrix, side information on first mode
X_covar2 a matrix, side information on second mode
X_covar3 a matrix, side information on third mode
W a list of orthogonal factor matrices - one for each mode, with the number of columns given by core_shape
G an array, core tensor with size specified by core_shape
C_ts an array, coefficient tensor, Tucker product of G,A,B,C
U an array, linear predictor,i.e. Tucker product of C_ts,X_covar1,X_covar2,X_covar3
seed = 34 dist = 'binary' data=sim_data(seed, whole_shape = c(20,20,20), core_shape=c(3,3,3), p=c(5,5,5),dist=dist, dup=5, signal=4)seed = 34 dist = 'binary' data=sim_data(seed, whole_shape = c(20,20,20), core_shape=c(3,3,3), p=c(5,5,5),dist=dist, dup=5, signal=4)
Supervised tensor decomposition with interactive side information on multiple modes. Main function in the package. The function takes a response tensor, multiple side information matrices, and a desired Tucker rank as input. The output is a rank-constrained M-estimate of the core tensor and factor matrices.
tensor_regress( tsr, X_covar1 = NULL, X_covar2 = NULL, X_covar3 = NULL, core_shape, niter = 20, cons = c("non", "vanilla", "penalty"), lambda = 0.1, alpha = 1, solver = "CG", dist = c("binary", "poisson", "normal"), traj_long = FALSE, initial = c("random", "QR_tucker") )tensor_regress( tsr, X_covar1 = NULL, X_covar2 = NULL, X_covar3 = NULL, core_shape, niter = 20, cons = c("non", "vanilla", "penalty"), lambda = 0.1, alpha = 1, solver = "CG", dist = c("binary", "poisson", "normal"), traj_long = FALSE, initial = c("random", "QR_tucker") )
tsr |
response tensor with 3 modes |
X_covar1 |
side information on first mode |
X_covar2 |
side information on second mode |
X_covar3 |
side information on third mode |
core_shape |
the Tucker rank of the tensor decomposition |
niter |
max number of iterations if update does not convergence |
cons |
the constraint method, "non" for without constraint, "vanilla" for global scale down at each iteration, "penalty" for adding log-barrier penalty to object function |
lambda |
penalty coefficient for "penalty" constraint |
alpha |
max norm constraint on linear predictor |
solver |
solver for solving object function when using "penalty" constraint, see "details" |
dist |
distribution of the response tensor, see "details" |
traj_long |
if "TRUE", set the minimal iteration number to 8; if "FALSE", set the minimal iteration number to 0 |
initial |
initialization of the alternating optimiation, "random" for random initialization, "QR_tucker" for deterministic initialization using tucker decomposition |
Constraint penalty adds log-barrier regularizer to
general object function (negative log-likelihood). The main function uses solver in function "optim" to
solve the objective function. The "solver" passes to the argument "method" in function "optim".
dist specifies three distributions of response tensor: binary, poisson and normal distribution.
If dist is set to "normal" and initial is set to "QR_tucker", then the function returns the results after initialization.
a list containing the following:
W a list of orthogonal factor matrices - one for each mode, with the number of columns given by core_shape
G an array, core tensor with the size specified by core_shape
C_ts an array, coefficient tensor, Tucker product of G,A,B,C
U linear predictor,i.e. Tucker product of C_ts,X_covar1,X_covar2,X_covar3
lglk a vector containing loglikelihood at convergence
sigma a scalar, estimated error variance (for Gaussian tensor) or dispersion parameter (for Bernoulli and Poisson tensors)
violate a vector listing whether each iteration violates the max norm constraint on the linear predictor, 1 indicates violation
seed = 34 dist = 'binary' data=sim_data(seed, whole_shape = c(20,20,20), core_shape=c(3,3,3), p=c(5,5,5),dist=dist, dup=5, signal=4) re = tensor_regress(data$tsr[[1]],data$X_covar1,data$X_covar2,data$X_covar3, core_shape=c(3,3,3),niter=10, cons = 'non', dist = dist,initial = "random")seed = 34 dist = 'binary' data=sim_data(seed, whole_shape = c(20,20,20), core_shape=c(3,3,3), p=c(5,5,5),dist=dist, dup=5, signal=4) re = tensor_regress(data$tsr[[1]],data$X_covar1,data$X_covar2,data$X_covar3, core_shape=c(3,3,3),niter=10, cons = 'non', dist = dist,initial = "random")
An S4 class for a tensor with arbitrary number of modes. The Tensor class extends the base "array" class to include additional tensor manipulation (folding, unfolding, reshaping, subsetting) as well as a formal class definition that enables more explicit tensor algebra.
number of modes (integer)
vector of modes (integer), aka sizes/extents/dimensions
actual data of the tensor, which can be 'array' or 'vector'
All of the decompositions and regression models in this package require a Tensor input.
James Li [email protected]
James Li, Jacob Bien, Martin T. Wells (2018). rTensor: An R Package for Multidimensional Array (Tensor) Unfolding, Multiplication, and Decomposition. Journal of Statistical Software, Vol. 87, No. 10, 1-31. URL: http://www.jstatsoft.org/v087/i10/.
Contracted (m-Mode) product between a Tensor of arbitrary number of modes and a list of matrices. The result is folded back into Tensor.
ttl(tnsr, list_mat, ms = NULL)ttl(tnsr, list_mat, ms = NULL)
tnsr |
Tensor object with K modes |
list_mat |
a list of matrices |
ms |
a vector of modes to contract on (order should match the order of |
Performs ttm repeated for a single Tensor and a list of matrices on multiple modes. For instance, suppose we want to do multiply a Tensor object tnsr with three matrices mat1, mat2, mat3 on modes 1, 2, and 3. We could do ttm(ttm(ttm(tnsr,mat1,1),mat2,2),3), or we could do ttl(tnsr,list(mat1,mat2,mat3),c(1,2,3)). The order of the matrices in the list should obviously match the order of the modes. This is a common operation for various Tensor decompositions such as CP and Tucker. For the math on the m-Mode Product, see Kolda and Bader (2009).
Tensor object with K modes
The returned Tensor does not drop any modes equal to 1.
T. Kolda, B. Bader, "Tensor decomposition and applications". SIAM Applied Mathematics and Applications 2009, Vol. 51, No. 3 (September 2009), pp. 455-500. URL: https://www.jstor.org/stable/25662308
tnsr <- new('Tensor',3L,c(3L,4L,5L),data=runif(60)) lizt <- list('mat1' = matrix(runif(30),ncol=3), 'mat2' = matrix(runif(40),ncol=4), 'mat3' = matrix(runif(50),ncol=5)) ttl(tnsr,lizt,ms=c(1,2,3))tnsr <- new('Tensor',3L,c(3L,4L,5L),data=runif(60)) lizt <- list('mat1' = matrix(runif(30),ncol=3), 'mat2' = matrix(runif(40),ncol=4), 'mat3' = matrix(runif(50),ncol=5)) ttl(tnsr,lizt,ms=c(1,2,3))
Contracted (m-Mode) product between a Tensor of arbitrary number of modes and a matrix. The result is folded back into Tensor.
ttm(tnsr, mat, m = NULL)ttm(tnsr, mat, m = NULL)
tnsr |
Tensor object with K modes |
mat |
input matrix with same number columns as the |
m |
the mode to contract on |
By definition, the number of columns in mat must match the mth mode of tnsr. For the math on the m-Mode Product, see Kolda and Bader (2009).
a Tensor object with K modes
The mth mode of tnsr must match the number of columns in mat. By default, the returned Tensor does not drop any modes equal to 1.
T. Kolda, B. Bader, "Tensor decomposition and applications". SIAM Applied Mathematics and Applications 2009, Vol. 51, No. 3 (September 2009), pp. 455-500. URL: https://www.jstor.org/stable/25662308
tnsr <- new('Tensor',3L,c(3L,4L,5L),data=runif(60)) mat <- matrix(runif(50),ncol=5) ttm(tnsr,mat,m=3)tnsr <- new('Tensor',3L,c(3L,4L,5L),data=runif(60)) mat <- matrix(runif(50),ncol=5) ttm(tnsr,mat,m=3)
The Tucker decomposition of a tensor. Approximates a K-Tensor using a n-mode product of a core tensor (with modes specified by ranks) with orthogonal factor matrices. If there is no truncation in all the modes (i.e. ranks = tnsr@modes), then this is the same as the HOSVD, hosvd. This is an iterative algorithm, with two possible stopping conditions: either relative error in Frobenius norm has gotten below tol, or the max_iter number of iterations has been reached. For more details on the Tucker decomposition, consult Kolda and Bader (2009).
tucker(tnsr, ranks = NULL, max_iter = 25, tol = 1e-05)tucker(tnsr, ranks = NULL, max_iter = 25, tol = 1e-05)
tnsr |
Tensor with K modes |
ranks |
a vector of the modes of the output core Tensor |
max_iter |
maximum number of iterations if error stays above |
tol |
relative Frobenius norm error tolerance |
Uses the Alternating Least Squares (ALS) estimation procedure also known as Higher-Order Orthogonal Iteration (HOOI). Intialized using a (Truncated-)HOSVD. A progress bar is included to help monitor operations on large tensors.
a list containing the following:
Zthe core tensor, with modes specified by ranks
Ua list of orthgonal factor matrices - one for each mode, with the number of columns of the matrices given by ranks
convwhether or not resid < tol by the last iteration
estestimate of tnsr after compression
norm_percentthe percent of Frobenius norm explained by the approximation
fnorm_residthe Frobenius norm of the error fnorm(est-tnsr)
all_residsvector containing the Frobenius norm of error for all the iterations
The length of ranks must match tnsr@num_modes.
T. Kolda, B. Bader, "Tensor decomposition and applications". SIAM Applied Mathematics and Applications 2009, Vol. 51, No. 3 (September 2009), pp. 455-500. URL: https://www.jstor.org/stable/25662308
tnsr <- rand_tensor(c(4,4,4,4)) tuckerD <- tucker(tnsr,ranks=c(2,2,2,2)) tuckerD$conv tuckerD$norm_percent plot(tuckerD$all_resids)tnsr <- rand_tensor(c(4,4,4,4)) tuckerD <- tucker(tnsr,ranks=c(2,2,2,2)) tuckerD$conv tuckerD$norm_percent plot(tuckerD$all_resids)
Unfolds the tensor into a matrix, with the modes in rs onto the rows and modes in cs onto the columns. Note that c(rs,cs) must have the same elements (order doesn't matter) as x@modes. Within the rows and columns, the order of the unfolding is determined by the order of the modes. This convention is consistent with Kolda and Bader (2009).
unfold(tnsr, row_idx, col_idx)unfold(tnsr, row_idx, col_idx)
tnsr |
the Tensor instance |
row_idx |
the indices of the modes to map onto the row space |
col_idx |
the indices of the modes to map onto the column space |
unfold(tnsr,row_idx=NULL,col_idx=NULL)
matrix with prod(row_idx) rows and prod(col_idx) columns
T. Kolda, B. Bader, "Tensor decomposition and applications". SIAM Applied Mathematics and Applications 2009, Vol. 51, No. 3 (September 2009), pp. 455-500. URL: https://www.jstor.org/stable/25662308.
tnsr <- rand_tensor() matT3<-unfold(tnsr,row_idx=2,col_idx=c(3,1))tnsr <- rand_tensor() matT3<-unfold(tnsr,row_idx=2,col_idx=c(3,1))