[1] P. Comon, ``Tensors: a brief introduction,'' IEEE Sig. Proc. Magazine, 31(3):44-53, May 2014. Preprint: hal-00923279

[1] P. Comon, X. Luciani, and A. L. F. de Almeida, ``Tensor Decompositions, Alternating Least Squares and other Tales,'' Jour. Chemometrics, 23:393--405, August 2009. Preprint: hal-00410057 or local preprint

In order to fix the ideas, let's take the case of 3rd order tensors. Once the three bases are chosen in each of the three vector spaces, the tensor is defined by a 3-way array. Its CP decomposition takes the form:

- Rank reduction.
Removing noise is standard in many applications, and can be
performed by truncating the multilinear rank. This rank is a triplet of
numbers, which correspond to the matrix rank of three possible
flattening matrices (obtained by storing the 3rd order array
values in matrix form); see [1] for complementary explanations
and bibliographical references. The reduction of the multilinear
rank implies (indirectly) tensor rank reduction. Denote G the new tensor obtained
from T by multilinear
rank reduction.

- Optimization. The next step consists of minimizing , i.e. a norm of the difference between actual tensor and model, for increasing values of r until the resulting difference is null. If the Frobenius norm is chosen, we minimize:

- Alternating Least Squares (ALS). It is the least attractive as far as performances are concerned. But it is the easiest to program.
- Gradient descent. It can be implemented with fixed or variable step size. In practice, computing the locally optimal step size is unuseful and unnecessarily costly.
- Quasi-Newton. The BFGS implementation is quite efficient.
- Levenberg-Marquardt. The implementation presented in [1] performs quite well.
- Conjugate gradient. This
is
the implementation of Fletcher-Reeves, with regular
reinitializations.

[2] M. Rajih, P. Comon, R. Harshman, ``Enhanced Line Search: A Novel Method to Accelerate Parafac,'' SIAM Journal on Matrix Analysis Appl., 30(3):1148--1171, September 2008. Preprint: hal-00327595 or local preprint

Downloads

Tensor package by Pierre Comon is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.

Based on a work at P.Comon's Tensor Package home page.

## General-purpose codes

• Most of the following codes
have been written or rewritten by X.Luciani during his
post-doc with P.Comon at I3S in 2009:

Tensor package by Pierre Comon is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.

Based on a work at P.Comon's Tensor Package home page.

- als.m (Alternating Least Squares)

- grad.m (Gradient descent)

- bfgs.m (Quasi Newton)

- lm.m (Levenberg-Marquardt)

- cg.m (Conjugate gradient)

- els.m (Enhanced Line Search, for any of the previous iterations)
- main.m (a main code, showing examples of how to call the
previous functions)

References:

[1] P. Comon, X. Luciani, and A. L. F. de Almeida, ``Tensor Decompositions, Alternating Least Squares and other Tales,'' Jour. Chemometrics, 23:393--405, August 2009. Preprint: hal-00410057

[2] M. Rajih, P. Comon, R. Harshman, ``Enhanced Line Search: A Novel Method to Accelerate Parafac,'' SIAM Journal on Matrix Analysis Appl., 30(3):1148--1171, September 2008. Preprint: hal-00327595

- SeROAP
- THOSVD
- CE
- DCPD

References:

[4] A. Pereira da Silva, P. Comon, and A. Lima Ferrer de Almeida, ``A finite algorithm to compute rank-1 tensor approximations,'' Sig. Proc. Letters, 23(7):959-963, July 2016. DOI: 10.1109/LSP.2016.2570862

- gradP.m (gradient descent)

- bfgsP.m (BFGS implementation of quasi-Newton)

- cgP.m (conjugate gradient)

[5] J.P. Royer, N. Thirion and P. Comon, ``Computing the polyadic decomposition of nonnegative third order tensors'', Signal Processing, Elsevier, vol.91, no.9, pp. 2159-2171, Sept. 2011. Preprint: hal-00618729.

• When tensors have a large size, we developed special codes able to compress the data (low rank) while at the same time maintain nonnegativity. These codes have been developed by J.E.Cohen and R.Cabral-Farias under the supervision of P.Comon:

Download the codes dedicated to big nonnegative tensors in a single zip archive

[6] J. E. Cohen, R. Cabral-Farias and P. Comon, ``Fast Decomposition of Large Nonnegative Tensors'', IEEE Sig. Proc. Letters, vol.22, no.7, pp.862-866, July 2015. Preprint: hal-01069069.

Download the codes to compute CP decompositions with flexible couplings.

[8] R. Cabral Farias, J. E. Cohen and P. Comon , ``Exploring multimodal data fusion through joint decompositions with flexible couplings'', IEEE Trans. Signal Processing, vol.64, no.18, pp.4830-4844, Sept. 2016. Preprints: hal-01158082, arxiv:1505.07717.

Download the codes to compute the CP decomposition when a factor matrix is structured, in a single zip archive.

[9] P. Comon, M. Sorensen, and E. Tsigaridas , ``Decomposing tensors with structured matrix factors reduces to rank-1 approximations'', ICASSP, Dallas, , March 14-19, 2010. Preprint: hal-00490248.

• Algorithms based on the joint characteristic function aiming at identifying blindly a linear system with more inputs than outputs: the CAF_toolbox, developed by X.Luciani, A.deAlmeida and P.Comon.

[10] X. Luciani, A. L. F. de Almeida and P. Comon, ``Blind Identiﬁcation of Underdetermined Mixtures Based on the Characteristic Function: The Complex Case'', IEEE Trans. Signal Processing, vol.59, no.2, pp.540-553, February 2011. Preprint: hal-00537838.

[11] J. E. Cohen, ``Environmental multiway data mining '', PhD thesis, Sept. 15, 2016, Univ. Grenoble Alpes. Reprint: tel-01371777

A main file calls one of the 3 functions below, depending of the type of tensor: symmetric, square non symmetric, or 3rd order with symmetric matrix slices:

- bornes (main)
- rangjs
- rangj3
- rgindscal3

[12] P. Comon and J. ten Berge, ``Generic and Typical Ranks of Three-Way Arrays,''

arxiv:0802.2371

[13] P. Comon, J. M. F. ten Berge, et al., ``Generic and Typical Ranks of Multi-Way Arrays,''

See also web sites of: Rasmus Bro, Three-mode company, Decotes project, Decoda project.