\documentclass[12pt]{article}

\setlength{\textwidth}{6.5in}
 \setlength{\textheight}{8.5in}
\setlength{\topmargin}{0.0in} \setlength{\oddsidemargin}{0.0in}

\setlength{\tabcolsep}{3pt}

\setcounter{tocdepth}{5}


\usepackage{graphicx}
\usepackage{curves}
\usepackage{alltt}

\begin{document}

\Huge

\begin{center}

\textbf{Combinatorial Data Analysis Miscellany: A (Nascent) Toolbox for MATLAB}

\end{center}

\bigskip

\huge

\noindent From the toolbox construction team (listed in alphabetical order):

\bigskip

 \noindent Lawrence Hubert; Hans-Friedrich K\"{o}hn; \mbox{Douglas} Steinley

 \bigskip

 \noindent M-files are available from:

 \smallskip

\large

 \verb+http://cda.psych.uiuc.edu/cda_miscellany_mfiles+

\Huge

 \bigskip

 \noindent Version: \today


\large

\newpage

\tableofcontents

\newpage

\listoftables

\listoffigures

\newpage

\section{Introduction}

Two previous MATLAB Toolboxes in this series have dealt with Cluster Analysis and
Unidimensional Scaling and the use of a single structure to help explain the
patterning of data within  a given proximity matrix.  A third Multistructural
Analysis Toolbox extended this interest in proximity matrix representation to the
additive combination of two (or more) of these structures chosen from the earlier
two Toolboxes. Here, we discuss a number of (diverse) areas under a rubric
labeled as the Combinatorial Data Analysis Miscellany Toolbox. As before,
we work entirely within a MATLAB environment and provide our
(open-source) M-files from the web site:

\begin{verbatim}
http://cda.psych.uiuc.edu/cda_miscellany_mfiles
\end{verbatim}

\noindent The Appendix lists the help-header comments for all the M-files mentioned throughout this manual.


\section{Unidimensional Scaling (Miscellany)}

This chapter includes two complete papers written by Hubert, Arabie, and Meulman.  The first, \emph{Linear Unidimensional Scaling in the $L_{2}$-Norm: Basic Optimization Methods Using MATLAB}, appeared in the \emph{Journal of Classification} (2002, \emph{19}, 303--328).  There is some overlap with the material presented in the Unidimensional Scaling Toolbox, but there is also enough unique to warrant its inclusion here more or less ``as is''.  The timings given in this paper for the Fortran and C implementations are also still very germane given the latest version of MATLAB (Release 2007a) and the use of the currently suggested Fortran Compiler (Intel Version 9.1).  Some minor editings and deletions from the original published form have been carried out.

\smallskip

The second paper, \emph{Multidimensional Scaling in the City-Block Metric: $L_{1}$ and $L_{2}$-Norm Optimization Methods Using MATLAB}, was intended as a companion piece to the \emph{JOC} article, but for a number of reasons, this never materialized.  The emphasis is on multiple unidimensional scalings in both the $L_{1}$ and $L_{2}$-Norm.  For the $L_{2}$-Norm, there is some overlap with material from the Multistructural Analysis Toolbox; the $L_{1}$-Norm discussion is unique to this paper, and in comparison to the $L_{2}$-Norm, it is of value to have this paper ``as is'' in the Miscellany Toolbox.

\subsection{Linear Unidimensional Scaling in the $L_{2}$-Norm: Basic Optimization Methods Using MATLAB}

The task of linear unidimensional scaling (LUS) can be characterized as a  specific data analysis problem: given a set of $n$ objects, $S = \{O_{1},...,O_{n}\}$, and an $n \times n$ symmetric proximity matrix $\mathbf{P} = \{p_{ij}\}$, arrange the objects along a single dimension such that the induced $n(n-1)/2$ interpoint distances between the objects reflect the proximities in $\mathbf{P}$.  The term ``proximity'' refers to any symmetric numerical measure of relationship between each object pair ($p_{ij} = p_{ji}$ for $1 \le i,j \le n$) and for which all self-proximities are considered irrelevant and set equal to zero ($p_{ii} = 0$ for $1 \le i \le n$).  As a technical convenience, proximities are assumed nonnegative and are given a dissimilarity interpretation, so that large proximities refer to dissimilar objects.

\smallskip

  As a starting point to be developed exclusively in this paper, we consider the most common formalization of measuring how close the interpoint distances are to the given proximities by the sum of squared discrepancies.  Specifically, we wish to find the $n$ coordinates, $x_{1}, x_{2}, \ldots, x_{n}$, such that the least-squares (or $L_{2}$) criterion
\begin{equation}
\sum_{i < j} (p_{ij} - |x_{j} - x_{i}|)^{2}
\end{equation}
is minimized.  Although there is some  arbitrariness in the selection of this measure of goodness-of-fit for metric scaling, the choice is traditional and has been discussed in some detail in the literature by Guttman (1968), Defays (1978), de Leeuw and Heiser (1977), and Hubert and Arabie (1986), among others. In the various sections that follow, several functions are presented within a MATLAB environment for this  $L_{2}$ task based on a number of different optimization strategies: dynamic programming; the iterative use of a quadratic assignment improvement heuristic; Pliner's technique of smoothing as implemented within a Guttman update method; and a nonlinear programming reformulation by Lau, Leung, and Tse (1998).

\smallskip

Besides making available the basic MATLAB M-functions for carrying out these various approaches to the unidimensional scaling task, several important computational improvements are also discussed and compared for the computationally intensive dynamic programming formulation that can guarantee a globally optimal solution.  These involve either transforming a given M-function with the MATLAB Compiler into C code that can in turn be submitted to a C/C++ compiler, or alternatively, rewriting an M-function and the mandatory MATLAB gateway directly in Fortran and then compiling into a MATLAB callable *.dll file (within a Windows environment).\footnote{Within the current version of MATLAB (Release 2007a), these external *.dll files now have the extension *.mexw32.}  In some cases studied, the computational improvements are very dramatic when the use of an external Fortran coded *.dll is compared either to one generated through C by use of the MATLAB Compiler, or as might be more expected, to the original interpreted M-function directly within a MATLAB environment.

\smallskip

In addition to developing the combinatorial optimization task of actually identifying a best unidimensional scaling, a later section introduces two additional problems within the LUS context: (a) the confirmatory fitting of a unidimensional scale (through coordinate estimation) based on a fixed (and given) object ordering; (b) the extension to nonmetric unidimensional scaling incorporating an additional optimal monotonic transformation of the proximities.  Both of these optimization tasks are formulated through the $L_{2}$ norm and carried out through applications of what is called the Dykstra-Kaczmarz method of solving linear (in)equality constrained least-squares tasks.


\subsubsection{LUS in the $L_{2}$-Norm}

As a reformulation of the $L_{2}$ unidimensional scaling task that will prove very convenient as a point of departure in our development of computational routines, the optimization suggested by (1) can be subdivided into two separate problems to be solved simultaneously: find a set of $n$ numbers, $x_{1} \le x_{2} \le \cdots \le x_{n}$, \emph{and} a permutation on the first $n$ integers, $\rho(\cdot) \equiv \rho$, for which
\begin{equation}
\sum_{i < j} (p_{\rho(i)\rho(j)} - (x_{j} - x_{i}))^{2}
\end{equation}
is minimized.  Thus, a set of locations (coordinates) is defined along a continuum as represented in ascending order by the sequence $x_{1}, x_{2}, \ldots, x_{n}$; the $n$ objects are allocated to these locations by the permutation $\rho$, so object $O_{\rho(i)}$ is placed at location $i$.  Without loss of generality we will impose the one additional constraint that $\sum_{i} x_{i} = 0$, i.e., any set of values, $x_{1}, x_{2}, \ldots, x_{n}$, can be replaced by $x_{1} - \bar{x}, x_{2} - \bar{x}, \ldots, x_{n} - \bar{x}$, where $\bar{x} = (1/n) \sum_{i} x_{i}$, without altering the value of (1) or (2).  Formally, if $\rho^{*}$ and $x_{1}^{*} \le x_{2}^{*} \le \cdots \le x_{n}^{*}$ define a global minimum of (2), and $\Omega$ denotes the set of all permutations of the first $n$ integers, then
\[ \sum_{i < j} (p_{\rho^{*}(i) \rho^{*}(j)} - (x_{j}^{*} - x_{i}^{*}))^{2}  =  \]
\[ \min[ \sum_{i < j} (p_{\rho(i) \rho(j)} - (x_{j} - x_{i}))^{2} \ | \ \rho \in \Omega; \ x_{1} \le \cdots \le x_{n}; \  \sum_{i} x_{i} = 0] . \]
The measure of loss in (2) can be reduced algebraically:
\begin{equation}
\sum_{i < j} p_{ij}^{2} + n(\sum_{i} x_{i}^{2} - 2\sum_{i} x_{i} t_{i}^{(\rho)}) ,
\end{equation}
subject to the constraints that $x_{1} \le \cdots \le x_{n}$ and $\sum_{i} x_{i} = 0$, and letting
\[t_{i}^{(\rho)} = (u_{i}^{(\rho)} - v_{i}^{(\rho)})/n , \]
\noindent where
\[ u_{i}^{(\rho)} = \sum_{j = 1}^{i - 1} p_{\rho(i) \rho(j)}
\  \mathrm{for} \ i \ge 2 ; \]
\[ v_{i}^{(\rho)} = \sum_{j = i+1}^{n} p_{\rho(i) \rho(j)} \ \mathrm{for} \  i < n , \]
\noindent and
\[ u_{1}^{(\rho)} = v_{n}^{(\rho)} = 0 .\]
\noindent In words, $u_{i}^{(\rho)}$ is the sum of the entries within row $\rho(i)$ of $\{p_{\rho(i) \rho(j)}\}$ from the extreme left up to the main diagonal; $v_{i}^{(\rho)}$ is the sum from the main diagonal to the extreme right.  Or, we might rewrite (3) as
\begin{equation}
\sum_{i < j} p_{ij}^{2} + n \left( \sum_{i} (x_{i} - t_{i}^{(\rho)})^{2} - \sum_{i} (t_{i}^{(\rho)})^{2} \right) .
\end{equation}
\noindent In (4), the two terms $\sum_{i} (x_{i} - t_{i}^{(\rho)})^{2}$ and $ \sum_{i} (t_{i}^{(\rho)})^{2}$ control the size of the discrepancy index since $ \sum_{i < j} p_{ij}^{2}$ is constant for any given data matrix.  Thus, to minimize the original index in (2), we should simultaneously minimize $\sum_{i} (x_{i} - t_{i}^{(\rho)})^{2}$ and maximize $ \sum_{i} (t_{i}^{(\rho)})^{2}$.  If the equivalent form of (3) is considered, our concern would be in minimizing $\sum_{i} x_{i}^{2}$ and maximizing $\sum_{i} x_{i} t_{i}^{(\rho)}$.


\smallskip


As noted first by Defays (1978), the minimization of (4) can be carried out directly by the maximization of the single term, $ \sum_{i} (t_{i}^{(\rho)})^{2}$ (under the mild regularity condition that all off-diagonal proximities in $\mathbf{P}$ are positive and not merely nonnegative).  Explicitly, if $\rho^{*}$ is a permutation that maximizes $ \sum_{i} (t_{i}^{(\rho)})^{2}$, then we can let $x_{i} = t_{i}^{(\rho^{*})}$, which eliminates the term  $\sum_{i} (x_{i} - t_{i}^{(\rho^{*})})^{2}$ from (4).  In short, because the order induced by $t_{1}^{(\rho^{*})}, \ldots, t_{n}^{(\rho^{*})}$ is consistent with the constraint $x_{1} \le x_{2} \le \cdots \le x_{n}$, the minimization of (4) reduces to the maximization of the single term $ \sum_{i} (t_{i}^{(\rho)})^{2}$, with the coordinate estimation completed as an automatic byproduct.

\smallskip


It is convenient to have a small numerical example available  as we discuss the various optimization strategies in the unidimensional scaling context.  To this end we list a data file in Table 1, called `\verb+number.dat+', that contains a dissimilarity matrix taken from Shepard, Kilpatric, and Cunningham (1975).  The stimulus domain is the first ten single-digits \{0,1,2, \ldots, 9\} considered as abstract concepts; the $10 \times 10$ proximity matrix (with an $i^{th}$ row or column corresponding to the $i-1$ digit) was constructed by averaging dissimilarity ratings for distinct pairs of those integers over a number of subjects and conditions.  Given the various analyses of this proximity matrix that have appeared in the literature (e.g., see Hubert, Arabie, and Meulman, 2001), the data reflect two types of very regular patterning based on absolute digit magnitude and the structural characteristics of the digits (e.g., the powers of 2 or of 3, the salience of the two additive/multiplicative identities [0/1], oddness/evenness).  These data will be relied on to provide concrete numerical illustrations of the various MATLAB functions we introduce, and will be loaded as a proximity matrix (and importantly, as one that is symmetric and has zero values along the main diagonal) in the MATLAB environment by the command \verb+`load number.dat'+.  As we will see in the various applications, the dominant single unidimensional scale found for these data is most consistent with digit magnitude.

\bigskip

\begin{table}

\caption{The number.dat data file from Shepard, Kilpatric, and Cunningham (1975)}

\begin{verbatim}
.000 .421 .584 .709 .684 .804 .788 .909 .821 .850
.421 .000 .284 .346 .646 .588 .758 .630 .791 .625
.584 .284 .000 .354 .059 .671 .421 .796 .367 .808
.709 .346 .354 .000 .413 .429 .300 .592 .804 .263
.684 .646 .059 .413 .000 .409 .388 .742 .246 .683
.804 .588 .671 .429 .409 .000 .396 .400 .671 .592
.788 .758 .421 .300 .388 .396 .000 .417 .350 .296
.909 .630 .796 .592 .742 .400 .417 .000 .400 .459
.821 .791 .367 .804 .246 .671 .350 .400 .000 .392
.850 .625 .808 .263 .683 .592 .296 .459 .392 .000
\end{verbatim}

\end{table}





\subsubsection{$L_{2}$-Norm Optimization Methods}

The subsections below emphasize four distinct optimization strategies for LUS in the $L_{2}$ norm.  We begin with the discussion of a dynamic programming strategy for the maximization of $\sum_{i} (t_{i}^{(\rho)})^{2}$ proposed by Hubert and Arabie (1986) that will produce globally optimal solutions for proximity matrices of sizes up to, say, the low twenties (within a MATLAB environment).  The maximization of $\sum_{i} (t_{i}^{(\rho)})^{2}$ over all permutations is a prototypical combinatorial optimization task, and unfortunately, representative of the class of so-called NP-hard problems (e.g., see Garey and Johnson, 1979); thus, any procedure yielding verifiably globally optimal solutions will be severely limited by the size of the matrices that could be realistically processed.   In providing MATLAB functions that carry out the dynamic programming optimization, mechanisms will also be discussed and evaluated for possibly speeding up the optimization process by the use of the MATLAB C/C++ Compiler and/or external Fortran subroutines and their gateways to allow  externally generated functions to be callable from MATLAB.\@ A later section shows how another well-known combinatorial optimization task, called quadratic assignment, can be used iteratively for LUS in the $L_{2}$ norm.  Based on the reformulation in (3), we concentrate on maximizing $\sum_{i} x_{i}t_{i}^{(\rho)}$, with iterative re-estimation of the coordinates, $x_{1}, \ldots, x_{n}$.  Various function implementations within MATLAB are given both for the basic quadratic assignment task as well as for  how it is used for LUS.\@ A third section presents gradient-based update strategies for LUS based on Guttman's original update strategy as modified by a smoothing technique due to Pliner (1996); and finally, a fourth section implements a (reformulated) nonlinear programming heuristic from Lau, Leung, and Tse (1998) that relies on the MATLAB Optimization Toolbox (and, in particular, its function \verb+fmincon.m+) for the MATLAB implementation that we give.



\paragraph{Dynamic Programming}

To maximize $\sum_{i} (t_{i}^{(\rho)})^{2}$ over all permutations, we construct a function, $\mathcal{F}(\cdot)$, by recursion for all possible subsets of the first $n$ integers, $\{1,2, \ldots, n\}$:

\smallskip

a) $\mathcal{F}(\oslash) = 0$, where $\oslash$ is the empty set;

\smallskip

b) $\mathcal{F}(R') = \max[\mathcal{F}(R) + d(R,i)]$, where $R'$ and $R$ are subsets of size $k+1$ and $k$, respectively; the maximum is taken over all subsets $R$ and indices $i$ such that $R' = R\cup\{i\}$; and $d(R,i)$ is the incremental value that would be added to the criterion \emph{if} the objects in $R$ \emph{had} formed the first $k$ values assigned by the optimal permutation and $i$ had been the next assignment made, i.e., $\rho(k+1) = i$.  Explicitly,
\[ d(R,i) = [(1/n)\{ \sum_{ j \in R } p_{ij} - \sum_{ j(\ne i) \notin R} p_{ij} \}]^{2}; \]

c) the optimal value of the criterion, $\mathcal{F}(\{1,2,\ldots,n\})$, is obtained for $R = \{1,2,\ldots,n\}$ and the optimal permutation, $\rho^{*}$, identified by working backwards through the recursion to identify the sequence of successive subsets of decreasing size that led to the value attained for $\mathcal{F}(\{1,2,\ldots,n\})$.

\medskip

This type of dynamic programming strategy is a very general one and can be used for any criterion for which the incremental value in identifying the index to be assigned to $\rho(k+1)$ does not depend on the particular \emph{order} of the assigned values in the set $\{\rho(1), \ldots, \rho(k)\}$.  The reader might refer to Hubert, Arabie, and Meulman (2001) for many more applications of dynamic programming in the combinatorial data analysis context.

\smallskip

In the three subsections that immediately follow, comparisons will be given between the execution times for four different implementations of the same dynamic programming strategy: the use of a raw M-file both under MATLAB Version 6.1 and Version 6.5, with the latter release now optimized for the execution of ``loops''; an M-function transformed to C code by the MATLAB C/C++ Compiler that is in turn compiled to produce a callable *.dll file within MATLAB; the use of a Fortran produced *.dll that is called upon to do the computationally intensive recursion within a MATLAB M-file.

\paragraph{The MATLAB Function uniscaldp.m}

The MATLAB function M-file, \verb+uniscaldp.m+,  carries out a unidimensional scaling of a symmetric proximity matrix (with a zero main diagonal and a dissimilarity interpretation) using the dynamic programming recursion just described.  The usage syntax has the form
\begin{verbatim}
[coord permut cumobfun diff] = uniscaldp(prox)
\end{verbatim}
where \verb+PROX+ is the input proximity matrix; \verb+COORD+ is the set of coordinates of the optimal unidimensional scaling in ascending order; \verb+PERMUT+ is the order of the objects in the optimal permutation (say, $\rho^{*}$); \verb+CUMOBFUN+ gives the cumulative values of the objective function for successive placements of the objects in the optimal permutation: $\sum_{i=1}^{k} (t_{i}^{(\rho^{*})})^{2}$ for $k = 1, \ldots, n$; \verb+DIFF+ is the value of the least-squares loss function for the optimal coordinates and object permutation.


\smallskip


  A recording of a MATLAB session using the \verb+number.dat+ data file follows (note that a semicolon is placed after the invocation of the M-function to initially suppress the output; transposes (') are then used on the output vectors to conserve space by only using row vectors in the listed output; also, to conserve space, blank lines are always deleted in any output given throughout the paper).  The three crucial outputs are:

(1) the optimal coordinates given as a response to listing \verb+coord'+:

-.6570, -4247, -.2608, -.1492, -.0566, .0842, .1988, .3258, .4050, .5345

(2) the optimal permutation given as a response to listing \verb+permut'+:

1 2 3 5 4 6 7 9 10 8

(3) the minimum value of the least-squares loss function in (1) given as a response to listing \verb+diff+:

1.959

\bigskip

\tiny



\begin{verbatim}
>> load number.dat
>> [coord permut cumobfun diff] = uniscaldp(number);
>> coord'
ans =
  Columns 1 through 6
   -0.6570   -0.4247   -0.2608   -0.1492   -0.0566    0.0842
  Columns 7 through 10
    0.1988    0.3258    0.4050    0.5345
>> permut'
ans =
     1     2     3     5     4     6     7     9    10     8
>> cumobfun'
ans =
  Columns 1 through 6
   43.1649   61.2019   68.0036   70.2296   70.5500   71.2590
  Columns 7 through 10
   75.2111   85.8257  102.2282  130.7972
>> diff
diff =
    1.9599
\end{verbatim}

\bigskip

\large


The second column of Table 2 provides some time comparisons (in seconds) for the use of \verb+uniscaldp.m+ over randomly constructed proximity matrices of size $n \times n$, for $n$ = 10 to 25.  The matrices were randomly generated using the utility program, \verb+ransymat.m+ (the usage syntax of \verb+ransymat.m+ can be seen from its header comments).  The computer on which these times were obtained is a laptop with a 1.7GHz Pentium processor and 1.0 GB of RAM; for matrices larger than size $25 \times 25$, an ``insufficient memory'' message was consistently obtained so the largest matrix size possible in Table 2 is 25 for the available RAM.  The execution times (obtained using the \verb+tic toc+ command pair in MATLAB) range from .05 seconds for $n$ = 10 to 3450.4 seconds for $n$ = 25.  As can be seen from the timings given, there is a fairly regular proportional increase in execution time of about 2.1 for each unit increase in $n$.


\paragraph{The MATLAB C/C++ Compiler}



One of the separate add-on components that can be obtained with \mbox{MATLAB} is a C/C++ Compiler that when applied to an M-file, such as \verb+uniscaldp.m+, produces C or C++ code.  The latter can itself then be compiled by a separate C/C++ compiler to produce (in a Windows environment) a *.dll file that can be called within MATLAB just like a *.m function.  As an example, we first renamed a version of \verb+uniscaldp.m+  to \verb+uniscaldpc.m+ and then applied in MATLAB 6.5 the C/C++ Compiler Version 3.0 with all the possible optimization options selected; the resulting code was compiled with the built-in C compiler (called \verb+lcc+) to produce \verb+uniscaldpc.dll+


\smallskip


We give the timings for the use of \verb+uniscaldpc.dll+ in the fourth column of Table 2.  As $n$ increases by 1, the execution times increase by a little more than twice, just as for \verb+uniscaldp.m+;  but overall (and rather incredibly, we might add) the compiled \verb+uniscaldpc.dll+ executes at about 7 to 8 times \emph{slower} than the interpreted file \verb+uniscaldp.m+.  The times are given for running \verb+uniscaldp.m+ under MATLAB 6.1 in the third column of Table 1 (but with the same computer).  Here, the execution times are a little less than four times slower than for the compiled C code.  So, it is pretty obvious that the accelerations provided in the new release of MATLAB (in 6.5) are working as promised, with a decrease in execution rate generally between 20 and 30 times for our recursive-intensive M-function.  At least for speed of execution considerations, these results more-or-less obviate the value of ever using the MATLAB C/C++ Compiler when MATLAB version 6.5 is available.


\paragraph{External Fortran Subroutines}

One strategy for decreasing the execution time of an M-file would replace part of the code that may be slow (usually nonvectorizable ``for'' loops, for example) by a call to a *.dll that is produced from a Fortran subroutine implementing the code in its own language.  As an example, \verb+uniscaldpf.m+, is a parallel of \verb+uniscaldp.m+ except that the actual recursion constructing the two crucial vectors (and from which the optimal solution is eventually identified by working backwards) is replaced by a call to \verb+uscalfor.dll+.  This latter file (called a Fortran MEX-file in MATLAB) was produced using the MATLAB Application Program Interface (API) through the Compaq Digital Visual Fortran (6.6) Compiler and with the central computational Fortran subroutine \verb+uscalfor.for+ and the second necessary gateway Fortran subroutine \verb+uscalforgw.for+.  These two subroutines are compiled together to produce \verb+uscalfor.dll+ that is then called in \verb+uniscaldpf.m+ to do ``the heavy lifting''.

In comparison to the other times listed in Table 2, the speedup provided by using the routine \verb+uniscaldpf.m+ is rather spectacular.  There is generally the same type of proportional increase in time that is slightly greater than twice for each unit change in $n$.  In comparison to the basic M-function, \verb+uniscaldp.m+, running under MATLAB 6.5, there is a 20/25 increase in execution speed for \verb+uniscaldpf.m+.

\bigskip

\begin{center}

\begin{table}

\caption{Time comparisons in seconds for the various implementations of unidimensional scaling through dynamic programming. The three entries marked by an asterisk are estimated values.}



\bigskip



\begin{tabular}{crrrr} \hline
 & & & & \\
matrix  & uniscaldp.m & uniscaldp.m & uniscaldpc.dll & uniscaldpf.m \\
size & (Version 6.5) & (Version 6.1) & & \\ [1ex] \hline
 & & & & \\
10 & .05 & .85 & .28 & .00 \\
11 & .11 & 1.86 & .54 & .00 \\
12 & .23 & 4.14 & 1.14 & .00 \\
13 & .46 & 9.29 & 2.50 & .02 \\
14 & .89 & 20.85 & 5.49 & .04 \\
15 & 1.81 & 46.50 & 12.43 & .08 \\
16 & 3.85 & 103.65 & 27.65 & .19 \\
17 & 8.23 & 228.84 & 61.30 & .39 \\
18 & 17.59 & 503.88 & 135.84 & .84 \\
19 & 37.62 & 1107.3 & 300.06 & 1.99 \\
20 & 80.10 & 2473.2 & 648.26 & 3.87 \\
21 & 170.60 & 5267.8 & 1421.1 & 8.15 \\
22 & 362.19 & 11420. & 3097.7 & 18.30 \\
23 & 767.93 & 24845. & 6772.0 & 40.40 \\
24 & 1625.3 & $54052.^*$ & 14985. & 85.99 \\
25 & 3450.4 &  $117595.^*$       &   $33158.^*$     & 185.93 \\ [1ex] \hline



\end{tabular}

\end{table}

\end{center}



\subsubsection{Iterative Quadratic Assignment}

Because of the manner in which the discrepancy index for the unidimensional scaling task can be rephrased as in (3) and (4), the two optimization subproblems to be solved simultaneously of identifying an optimal permutation and a set of coordinates can be separated:

\smallskip

(a)  assuming that an ordering of the objects is known (and denoted, say, as $\rho^{0}$ for the moment), find those values $x_{1}^{0} \le \cdots \le x_{n}^{0}$ to minimize $\sum_{i} (x_{i}^{0} - t_{i}^{(\rho^{0})})^{2}$.  If the permutation $\rho^{0}$ produces a \emph{monotonic} form for the matrix $\{p_{\rho^{0}(i) \rho^{0}(j)}\}$ in the sense that $t_{1}^{(\rho^{0})} \le t_{2}^{(\rho^{0})} \le \cdots \le t_{n}^{(\rho^{0})}$, the coordinate estimation is immediate by letting $x_{i}^{0} = t_{i}^{(\rho^{0})}$, in which case $\sum_{i} (x_{i}^{0} - t_{i}^{(\rho^{0})})^{2}$ is zero.


\smallskip

(b) assuming that the locations $x_{1}^{0} \le \cdots \le x_{n}^{0}$ are known, find the permutation $\rho^{0}$ to maximize $\sum_{i} x_{i} t_{i}^{(\rho^{0})}$.  We note from the work of Hubert and Arabie
(1986, p.\ 189) that any such permutation which even only locally maximizes $\sum_{i} x_{i} t_{i}^{(\rho^{0})}$, in the sense that no adjacently placed pair of  objects in $\rho^{0}$ could be interchanged to increase the index, will produce a monotonic form for the  non-negative matrix $\{p_{\rho^{0}(i) \rho^{0}(j)}\}$.  Also, the task of finding the permutation $\rho^{0}$ to maximize $\sum_{i} x_{i} t_{i}^{(\rho^{0})}$ is actually a quadratic assignment (QA) task which has been discussed extensively in the literature of operations research, e.g., see Francis and White (1974), Lawler (1975), Hubert and Schultz (1976), among others.  As usually defined, a QA problem involves two $n \times n$ matrices $\mathbf{A} = \{a_{ij}\}$ and $\mathbf{B} = \{b_{ij}\}$, and we seek a permutation $\rho$ to maximize \begin{equation}
\Gamma(\rho) = \sum_{i,j} a_{\rho(i) \rho(j)} b_{ij}.
\end{equation}
 If we define $b_{ij} = |x_{i} - x_{j}|$ and let $a_{ij} = p_{ij}$, then \[  \Gamma(\rho) = \sum_{i,j} p_{\rho(i) \rho(j)} |x_{i} - x_{j}| = 2n \sum_{i} x_{i} t_{i}^{(\rho)}, \] and thus, the permutation that maximizes $\Gamma(\rho)$ also maximizes $\sum x_{i} t_{i}^{(\rho)}$.


\smallskip

The QA optimization task as formulated through (5) has an enormous literature attached to it, and the reader is referred to Pardalos and \mbox{Wolkowicz} (1994) for an up-to-date and comprehensive review.  For current purposes and as provided in three general M-functions of the next section (\verb+pairwiseqa.m+, \verb+rotateqa.m+, and \verb+insertqa.m+), one might consider the optimization of (5) through simple object interchange/rearrangement heuristics.  Based on given  matrices $\mathbf{A}$ and $\mathbf{B}$, and beginning with some permutation (possibly chosen at random), local interchanges/rearrangements of a particular type are implemented until no improvement in the index can be made.  By repeatedly initializing such a process randomly, a distribution over a set of local optima can be achieved.
At least within the context of some common data analysis applications, such a distribution may be highly relevant diagnostically for explaining whatever structure might be inherent in the matrix $\mathbf{A}$.


\smallskip

In a subsequent subsection, we introduce the main M-function (\verb+uniscalqa.m+) for unidimensional scaling based on these earlier QA optimization strategies. In effect, we begin with an equally-spaced set of fixed coordinates with their interpoint distances defining the $\mathbf{B}$ matrix of the general QA index in (5)  and a random object permutation;  a locally-optimal permutation is then identified through a collection of local interchanges/rearrangements; the coordinates are re-estimated based on this identified permutation, and the whole process repeated until no change can be made in either the identified permutation or coordinate collection.

\paragraph{The QA Interchange/Rearrangement Heuristics}

The three M-functions that carry out general QA interchange/rearrangement heuristics all have the same general usage syntax (note the use of three dots to denote a statement continuation in MATLAB):

\bigskip

\begin{verbatim}

[outperm rawindex allperms index] = pairwiseqa(prox,targ,inperm)

[outperm rawindex allperms index] = ...
                        rotateqa(prox,targ,inperm,kblock)

[outperm rawindex allperms index] = ...
                        insertqa(prox,targ,inperm,kblock)

\end{verbatim}

\bigskip

\noindent \verb+pairwiseqa.m+  carries out an iterative QA maximization task using the
pairwise interchanges of objects in the current permutation defining the row and column
order of the data matrix.  All possible such interchanges are generated and considered in turn, and whenever an increase in the cross-product index would result from a particular interchange, it is made immediately.  The process continues until the current permutation cannot be improved upon by any such pairwise object interchange; this final locally optimal permutation is \verb+OUTPERM+.
The input beginning permutation is \verb+INPERM+ (a permutation of the first $n$ integers);
\verb+PROX+ is the $n \times n$ input proximity matrix and
\verb+TARG+ is the $n \times n$ input target matrix (which are respective  analogues of the matrices $\mathbf{A}$ and $\mathbf{B}$ of (5));
the final \verb+OUTPERM+ row and column permutation of \verb+PROX+ has the cross-product index \verb+RAWINDEX+ with respect to \verb+TARG+.  The cell array
\verb+ALLPERMS+ contains \verb+INDEX+ entries corresponding to all the
permutations identified in the optimization, from
 \verb+ALLPERMS{1} = INPERM + to
 \verb+ALLPERMS{INDEX} = OUTPERM+.
  (Note that within a MATLAB environment, entries of a cell array must be accessed through the curly braces, \{  \}.)
\verb+rotateqa.m+ carries out a similar iterative QA maximization task but now uses the
rotation (or inversion) of from 2 to \verb+KBLOCK+ (which is less than or equal to $n-1$) consecutive objects in
the current permutation defining the row and column order of the data matrix.  \verb+insertqa.m+ relies on the (re-)insertion of from 1 to \verb+KBLOCK+  consecutive objects somewhere in
the permutation defining the current row and column order of the data matrix.

\paragraph{The MATLAB Function uniscalqa.m}

The MATLAB M-file, \verb+uniscalqa.m+, carries out a unidimensional scaling of a symmetric dissimilarity matrix (with a zero main diagonal) using an iterative quadratic assignment strategy.  We begin with an equally-spaced target, a (random) starting permutation, and use a sequential combination of the pairwise interchange/rotation/insertion heuristics; the target matrix is re-estimated based on the identified (locally optimal) permutation.  The whole process is repeated until no changes can be made in the target or the identified (locally optimal) permutation.  The explicit usage syntax is
\begin{verbatim}
[outperm rawindex allperms index coord diff] = ...
                   uniscalqa(prox,targ,inperm,kblock)
\end{verbatim}
where all terms are present either in \verb+uniscaldp.m+ or in the three QA heuristic M-functions of the previous subsection.  A recording of a MATLAB session using \verb+number.dat+ follows with results completely consistent with what was identified using \verb+uniscaldp.m+. Note the application of the built-in MATLAB function \verb+randperm(10)+ to obtain a random input permutation of the first 10 digits, and the use of the utility M-function, \verb+ransymat(10)+, to generate a target matrix \verb+targlin+ based on an equally (and unit) spaced set of coordinates.  Also, the optimal permutation given in response to \verb+outperm+ is the (equivalent) reversal of that given earlier using \verb+uniscaldp.m+; thus, the optimal coordinates given in response to \verb+coord'+ are listed in the reverse order as well.

\bigskip

\tiny

\begin{verbatim}
>> load number.dat
>> [prox10 targlin targcir] = ransymat(10);
>> inperm = randperm(10);
>> kblock = 2;
>> [outperm rawindex allperms index coord diff] = ...
uniscalqa(number,targlin,inperm,kblock);
>> outperm
outperm =
     8    10     9     7     6     4     5     3     2     1
>> coord'
ans =
  Columns 1 through 6
   -0.5345   -0.4050   -0.3258   -0.1988   -0.0842    0.0566
  Columns 7 through 10
    0.1492    0.2608    0.4247    0.6570
>> diff
diff =
    1.9599
\end{verbatim}

\large

\subsubsection{Gradient-Based Optimization with Pliner's Smoothing Strategy}

In Guttman's 1968 paper on multidimensional scaling, the optimization task in (1) is treated as a special case of a general iterative algorithm based on the partial derivatives of (1) with respect to the unknown locations.  For one dimension, Guttman's multidimensional scaling algorithm reduces to a simple updating procedure:
\begin{equation}
x_{i}^{(t+1)} = \frac{1}{n} \sum_{j=1}^{n} p_{ij} \mathrm{sign}(x_{i}^{(t)} - x_{j}^{(t)}),
\end{equation}
where $t$ is the index of iteration.  As pointed out by de Leeuw and Heiser (1977), convergence of (6) is guaranteed because $x_{i}^{(t+1)}$ only depends on the rank order of $x_{1}^{(t)},\ldots,
x_{n}^{(t)}$, there are a finite number of different rank orders, and no rank order can be repeated with intermediate different rank orders.  In fact, the stationary points of (6) are defined by all possible orderings of $\mathbf{P}$ that lead to monotonic forms.  Specifically, if $x_{1},\ldots,x_{n}$ is a stationary point of (6) and $\rho$ is the permutation for which $x_{\rho(1)} \le x_{\rho(2)} \le \cdots \le x_{\rho(n)}$, then $\{p_{\rho(i)\rho(j)}\}$ is monotonic, i.e., $t_{1}^{(\rho)} \le \cdots \le t_{n}^{(\rho)}$, and $t_{i}^{(\rho)} = x_{\rho(i)}$ for $1 \le i \le n$.  Conversely, if $\{p_{ij}\}$ is monotonic, then (6) converges in one step if we let the initial value of $x_{i}$ be, say, $i$ for $1 \le i \le n$.


\smallskip

Guttman's updating algorithm is in reality a procedure for finding mono\-tonic forms for a proximity matrix and only very indirectly can it even be characterized as a strategy for unidimensional scaling.  From a somewhat wider perspective, the general weakness of the monotonic forms for a given matrix may indicate why multidimensional scaling methods generally have such difficulties with local optima when restricted to a single dimension (e.g., see Shepard, 1974, pp.\ 378--379).  As can be seen in the way the index of goodness-of-fit is rewritten in (4), the crucial quantity for distinguishing among different monotonic forms is $\sum_{i} (t_{i}^{(\rho)})^{2}$.  Consideration of this latter term disappears in Guttman's update method because of the algorithm's reliance on a gradient approach; but as we will see momentarily, there is a method for improving  upon the  basic update strategy enormously through a smoothing strategy.

\paragraph{The MATLAB Function guttorder.m}

The MATLAB M-function, \verb+guttorder.m+, carries out a unidimensional scaling of a symmetric proximity matrix based on the Guttman update formula in (6).  The usage syntax is
\begin{verbatim}
[gcoordsort gperm] = guttorder(prox,inperm)
\end{verbatim}
where \verb+PROX+ and \verb+INPERM+ are as before, and the output vector \verb+GCOORDSORT+ contains the coordinates ordered from the most negative to most positive; \verb+GPERM+ is the object permutation indicating where the objects are placed at the ordered coordinates in \verb+GCOORDSORT+.  One easy exercise for the reader would be to call \verb+guttorder+ with \verb+inperm+ as \verb+randperm(10)+ and \verb+prox+ as \verb+number+ and merely use the `up arrow' key to retrieve the call to \verb+guttorder+ and rerun the routine with a new random starting permutation.  One will quickly see the weakest of the update procedure in (6) in finding anything that isn't just another local optimum.


\paragraph{Pliner's Smoothing Strategy and the MATLAB Function plinorder.m}

Although the use of the basic Guttman update formula is severely prone to finding only local optima, a smoothing strategy applied to (6) seems to alleviate this problem (almost) completely.  Very simply, Pliner's (1996) smoothing strategy for the sign function would replace $\mathrm{sign}(t)$ in (6) with
\[ \begin{array}{ll}
(t/\epsilon)(2 - [|t|/\epsilon]) & \mbox{if $|t| \le \epsilon$}; \\
\mbox{sign $(t)$}                         &   \mbox{if $|t| > \epsilon$},
\end{array}
\]
for $\epsilon > 0$.  Beginning with a randomly generated set of initial coordinate values and a sufficiently large value of $\epsilon$
(e.g., in the M-function, \verb+plinorder.m+, introduced below, we use Pliner's suggestion of an initial value of $\epsilon$ equal to twice the maximum of the row (or column) averages of the input proximity matrix), the update in (6) (with the replacement smoother) would be applied until convergence.  The parameter $\epsilon$ (given as \verb+ep+ in the M-function) is then reduced (e.g., we use \verb>ep = ep*(100-k+1)/100> for \verb+k = 2:100+), and beginning with the coordinates from the previous solution, the update in (6) is again applied until convergence.  The process continues until $\epsilon$ has been effectively reduced to zero.

\smallskip

Pliner's strategy is a relatively simple modification in the use of the iterative update in (6), and although it is still a heuristic strategy in the sense that a globally optimal solution is not guaranteed, the authors' experience with it suggests that it works incredibly well.  (We might also add that because of its computational simplicity and speed of execution, it may be the key to scaling huge proximity matrices.)  The M-function, \verb+plinorder.m+, has the usage syntax as follows:
\begin{verbatim}
[pcoordsort pperm gcoordsort gperm gdiff pdiff] = ...
                                 plinorder(prox,inperm)
\end{verbatim}
where some of the terms are the same as in \verb+guttorder.m+ since that update method is initially repeated with the invocation of \verb+plinorder.m+; \verb+PCOORDSORT+ and \verb+PPERM+ are analogues of \verb+GCOORDSORT+ and \verb+GPERM+ but using the smoother, and \verb+PDIFF+ and \verb+GDIFF+ are the least-squares loss function values for using the Pliner smoother and the Guttman update, respectively.  The pattern illustrated by the single call of \verb+plinorder.m+ to follow is expected: the smoothing strategy identifies a globally optimal solution and the Guttman update provides one that is only locally optimal.

\bigskip

\tiny

\begin{verbatim}
>> load number.dat
>> inperm = randperm(10);
>> [pcoordsort pperm gcoordsort gperm gdiff pdiff] = ...
plinorder(number,inperm);
>> pcoordsort'
ans =
  Columns 1 through 6
   -0.6570   -0.4247   -0.2608   -0.1492   -0.0566    0.0842
  Columns 7 through 10
    0.1988    0.3258    0.4050    0.5345
>> pperm
pperm =
     1     2     3     5     4     6     7     9    10     8
>> gcoordsort'
ans =
  Columns 1 through 6
   -0.5089   -0.3784   -0.2160   -0.1426   -0.1110    0.1316
  Columns 7 through 10
    0.1735    0.2902    0.3502    0.4114
>> gperm
gperm =
     2     6     5     9     1    10     8     4     3     7
>> gdiff
gdiff =
    5.9895
>> pdiff
pdiff =
    1.9599
\end{verbatim}

\large

\subsubsection{A Nonlinear Programming Heuristic}

In considering the unidimensional scaling task in (1), Lau, Leung, and Tse (1998) note the  equivalence to the minimization over $x_{1}, x_{2}, \ldots, x_{n}$ of
\begin{equation}
\sum_{i < j} \min \{[p_{ij} - (x_{i} - x_{j})]^{2}, [p_{ij} - (x_{j} - x_{i})]^{2}\}.
\end{equation}
Two zero/one variables can then be defined, $w_{1ij}$ and $w_{2ij}$, and (7) rewritten as the mathematical program
\begin{equation}
 \mbox{minimize} \sum_{i < j} \{w_{1ij}(e_{1ij})^{2} + w_{2ij}(e_{2ij})^{2}\}
\end{equation}
subject to
\[ p_{ij} = x_{i} - x_{j} + e_{1ij}; \]
\[ p_{ij} = x_{j} - x_{i} + e_{2ij}; \]
\[ w_{1ij} + w_{2ij} = 1; \]
\[ w_{1ij}, w_{2ij} \ge 0, \]
where $e_{1ij}$ is the error if $x_{i} > x_{j}$ and $e_{2ij}$ is the error if $x_{i} < x_{j}$.  These authors observe that the binary restriction on $w_{1ij}$ and $w_{2ij}$ can be removed since they will automatically be forced to zero or one.  In short, what initially appears as a combinatorial optimization task in (2) has now been replaced by a nonlinear programming model in (8).

\paragraph{The MATLAB Function unifitl2nlp.m}

The M-function, \verb+unifitl2nlp.m+, carries out the optimization task specified in (8) by a call to a very general M-function from the MATLAB Optimization Toolbox, \verb+fmincon.m+.  The latter is an extremely general routine for the minimization of a constrained multivariable function, and requires in our case a separate M-function, \verb+objfunl2.m+, to evaluate the objective function in (8).  So, to use the function \verb+unifitl2nlp.m+, the user needs to have the Optimization Toolbox installed.  The usage syntax for \verb+unifitl2nlp.m+ has the form
\begin{verbatim}
[startcoord begval outcoord endval exitflag] = ...
                              unifitl2nlp(prox,inperm)
\end{verbatim}
An input permutation \verb+INPERM+ is used to obtain a set of starting coordinates (\verb+STARTCOORD+) that would lead to an initial least-squares loss value (\verb+BEGVAL+).  The starting coordinates are obtained from the usual $t_{i}^{(\rho)}$ formula of (3) irrespective of whether \verb+INPERM+ provides a monotonic form for the reordered matrix \verb+PROX(INPERM,INPERM)+ or not.  The ending coordinates (\verb+OUTCOORD+) at the end of the process leads to a final least-squares loss value (\verb+ENDVAL+).  The \verb+EXITFLAG+ variable gives the success of the optimization (greater than 0 indicates convergence; $0$ implies that the maximum number of function evaluations or iterations were reached; less than 0 denotes nonconvergence).


\smallskip


An example of the use of \verb+unifitl2nlp.m+ is given below for two starting permutations --- the identity ordering and the second one random.  Given these results and others that the reader can replicate given the availability of the M-function, it appears in general that ``the apple is not allowed to fall very far from the tree''.  The end result is very close to where one starts, which is very similar to the disappointing performance of an unmodified Guttman update strategy.  The need to have such a good initial permutation to start with,  pretty much defeats the use of the nonlinear programming reformulation as a search technique.  Both iterative QA and Pliner's smoother, which can begin just with random permutations and usually end up with very good final permutations, would appear thus far to be the heuristic methods of choice.

\bigskip

\tiny


\begin{verbatim}
>> load number.dat
>> inperm = [1 2 3 4 5 6 7 8 9 10];
>> [startcoord begval outcoord endval exitflag] = ...
unifitl2nlp(number,inperm);
>> outcoord'
ans =
  Columns 1 through 6
   -0.6570   -0.4247   -0.2608   -0.1392   -0.0666    0.0842
  Columns 7 through 10
    0.1988    0.3627    0.4058    0.4968
>> endval
endval =
    2.1046
>> inperm = randperm(10);
>> [startcoord begval outcoord endval exitflag] = ...
unifitl2nlp(number,inperm);
>> outcoord'
ans =
  Columns 1 through 6
    0.2094   -0.0955    0.2728    0.0100   -0.3778    0.2434
  Columns 7 through 10
    0.0312   -0.3061   -0.4842    0.4968
>> endval
endval =
    5.9856
\end{verbatim}

\large

\subsubsection{The Implementation of Nonmetric and Confirmatory LUS}


In developing linear unidimensional scaling (as well as other types of) representations for a proximity matrix, it is convenient to have a general mechanism available for solving linear (in)equality constrained least-squares tasks.  The two such instances discussed in this section involve (a) the confirmatory fitting of a given object order to a proximity matrix (through an M-file called \verb+linfit.m+), and (b) the construction of an optimal monotonic transformation of a proximity matrix in relation to a given unidimensional ordering (through an M-file called \verb+proxmon.m+).  In both of these cases, we rely on what can be called the Dykstra-Kaczmarz method.  An equality constrained least-squares task may be rephrased as a linear system of equations, with the later solvable through a strategy of iterative projection as attributed to Kaczmarz (1937; see Bodewig, 1956, pp.\ 163--164); a more general inequality constrained least-squares task can also be approached through iterative projection as developed by Dykstra (1983).  The Kaczmarz and Dykstra strategies are implemented within the two M-files, \verb+linfit.m+ and \verb+proxmon.m+, discussed below.

\paragraph{The Confirmatory Fitting of a Given Order Using the MATLAB Function linfit.m}

The MATLAB M-function, \verb+linfit.m+, fits a set of coordinates to a given proximity matrix based on some given input permutation, say, $\rho^{(0)}$.  Specifically, we seek $x_{1} \le x_{2} \le \cdots \le x_{n}$ such that $\sum_{i < j} (p_{\rho^{0}(i) \rho^{0}(j)} - |x_{j} - x_{i}|)^{2}$ is minimized (and where the permutation $\rho^{(0)}$ may not even put the matrix $\{p_{\rho^{0}(i) \rho^{0}(j)}\}$ into a monotonic form).  Using the syntax
\begin{verbatim}
[fit diff coord] = linfit(prox,inperm)
\end{verbatim}
the matrix $\{|x_{j} - x_{i}|\}$ is referred to as the fitted matrix (\verb+FIT+); \verb+COORD+ gives the ordered coordinates; and \verb+DIFF+ is the value of the least-squares criterion. The fitted matrix is found through the Dykstra-Kaczmarz method where the equality constraints defined by distances along a continuum are imposed to find the fitted matrix, i.e., if $i < j < k$, then $|x_{i} - x_{j}| + |x_{j} - x_{k}| = |x_{i} - x_{k}|$.  Once found, the actual ordered coordinates are retrieved by the usual $t_{i}^{(\rho^{0})}$ formula in (3) but computed on \verb+FIT+.

\smallskip

The example below of the use of \verb+linfit.m+  fits two separate orders: the identity permutation and the one that we know is least-squares optimal.  The consistency of the results can be compared to those given earlier.

\bigskip

\tiny

\begin{verbatim}
>> load number.dat
>> inperm = [1 2 3 4 5 6 7 8 9 10];
>> [fit diff coord] = linfit(number,inperm);
>> coord'
ans =
  Columns 1 through 6
   -0.5345   -0.4050   -0.3258   -0.1988   -0.0842    0.0566
  Columns 7 through 10
    0.1492    0.2608    0.4247    0.6570
>> diff
diff =
    2.1046
>> inperm = [1 2 3 5 4 6 7 9 10 8];
>> [fit diff coord] = linfit(number,inperm);
>> coord'
ans =
  Columns 1 through 6
   -0.6570   -0.4247   -0.2608   -0.1492   -0.0566    0.0842
  Columns 7 through 10
    0.1988    0.3258    0.4050    0.5345
>> diff
diff =
    1.9599
\end{verbatim}

\large


\paragraph{The Monotonic Transformation of a Proximity Matrix Using the MATLAB Function proxmon.m}



The MATLAB function, \verb+proxmon.m+, provides a mono\-ton\-ically transformed proximity matrix that is close in a least-squares sense to a given input matrix.  The syntax is
\begin{verbatim}
[monproxpermut vaf diff] = proxmon(proxpermut,fitted)
\end{verbatim}
Here, \verb+PROXPERMUT+ is the input proximity matrix (which may have been subjected to an initial row/column permutation, hence the suffix `\verb+PERMUT+') and \verb+FITTED+ is a given target matrix;  the output matrix \verb+MONPROXPERMUT+ is closest to \verb+FITTED+ in a least-squares sense and obeys the order constraints obtained from each pair of entries in (the upper-triangular portion of) \verb+PROXPERMUT+ (and where the inequality constrained optimization is carried out using the Dykstra-Kaczmarz iterative projection strategy);  \verb+VAF+ denotes `var\-i\-ance-\-ac\-coun\-ted-\-for' and indicates how much variance in \verb+MONPROXPERMUT+ can be accounted for by \verb+FITTED+; finally \verb+DIFF+ is the value of the least-squares loss function and is (one-half)  the sum of squared differences between the entries in \verb+FITTED+ and \verb+MONPROXPERMUT+.

\smallskip

In the notation of the previous section when fitting a given order, \verb+FITTED+ would correspond to the matrix $\{|x_{j} - x_{i}|\}$, where $x_{1} \le x_{2} \le \cdots \le x_{n}$; the input \verb+PROXPERMUT+ would be $\{p_{\rho^{0}(i) \rho^{0}(j)}\}$; \verb+MONPROXPERMUT+ would be $\{f(p_{\rho^{0}(i) \rho^{0}(j)})\}$, where the function $f(\cdot)$ satisfies the monotonicity constraints, i.e., if
$p_{\rho^{0}(i) \rho^{0}(j)} < p_{\rho^{0}(i') \rho^{0}(j')}$ for $1 \le i  < j \le n$ and $1 \le i' < j' \le n$, then $f(p_{\rho^{0}(i) \rho^{0}(j)}) \le f(p_{\rho^{0}(i') \rho^{0}(j')})$.  The transformed proximity matrix $\{f(p_{\rho^{0}(i) \rho^{0}(j)})\}$ minimizes the least-squares criterion (\verb+DIFF+) of
\[ \sum_{i < j} (f(p_{\rho^{0}(i) \rho^{0}(j)}) - |x_{j} - x_{i}|)^{2} , \]
over all functions $f(\cdot)$ that satisfy the monotonicity constraints.  The \verb+VAF+ is a normalization of this loss value by the sum of squared deviations of the transformed proximities from their mean:
\[ \mbox{VAF} = 1 - \frac{\sum_{i < j} (f(p_{\rho^{0}(i) \rho^{0}(j)}) - |x_{j} - x_{i}|)^{2}}{\sum_{i < j} (f(p_{\rho^{0}(i) \rho^{0}(j)}) - \bar{f})^{2}} , \]
where $\bar{f}$ denotes the mean of the off-diagonal entries in $\{f(p_{\rho^{0}(i) \rho^{0}(j)})\}$.


\paragraph{An Application Incorporating proxmon.m}

The script M-file listed below gives an application of \verb+proxmon.m+ using the globally optimal permutation found previously for our \verb+number.dat+ matrix.  First, \verb+linfit.m+ is invoked to obtain a fitted matrix (\verb+fit+); \verb+proxmon.m+ then generates the monotonically transformed proximity matrix (\verb+monproxpermut+) with \verb+vaf+ = .5821 and \verb+diff+ = 1.0623.  The strategy is then repeated cyclically (i.e., finding a fitted matrix based on the monotonically transformed proximity matrix, finding a new monotonically transformed matrix, and so on).  To avoid degeneracy (where all matrices would converge to zeros), the sum of squares of the fitted matrix is kept the same as it was initially; convergence is based on observing a minimal change (less than 1.0e-006) in the \verb+vaf+.  As indicated in the output below, the final \verb+vaf+ is .6672 with a \verb+diff+ of .9718. (Although the globally optimal permutation found earlier for \verb+number.dat+ remains the same throughout the construction of the optimal monotonic transformation, in this particular example it would also remain optimal with the same vaf if the unidimensional scaling was repeated with \verb+monproxpermut+ now considered the input proximity matrix.  Even though probably rare, other data sets might not have such an invariance, and it may be desirable to initiate an iterative routine that finds  both a unidimensional scaling [i.e., an object ordering] in addition to monotonically transforming the proximity matrix.)

\bigskip

\tiny

\begin{verbatim}
load number.dat
inperm = [8 10 9 7 6 4 5 3 2 1];
[fit diff coord] = linfit(number,inperm);
[monproxpermut vaf diff] = ...
    proxmon(number(inperm,inperm),fit);
sumfitsq = sum(sum(fit.^2));
prevvaf = 2;
while (abs(prevvaf-vaf) >= 1.0e-006)
   prevvaf = vaf;
   [fit diff coord] = linfit(monproxpermut,1:10);
   sumnewfitsq = sum(sum(fit.^2));
   fit = sqrt(sumfitsq)*(fit/sqrt(sumnewfitsq));
    [monproxpermut vaf diff] = proxmon(number(inperm,inperm), fit);
end

fit
diff
coord'
monproxpermut
vaf
\end{verbatim}

\begin{verbatim}
fit =
  Columns 1 through 6
         0    0.0824    0.1451    0.3257    0.4123    0.5582
    0.0824         0    0.0627    0.2432    0.3298    0.4758
    0.1451    0.0627         0    0.1806    0.2672    0.4131
    0.3257    0.2432    0.1806         0    0.0866    0.2325
    0.4123    0.3298    0.2672    0.0866         0    0.1459
    0.5582    0.4758    0.4131    0.2325    0.1459         0
    0.5834    0.5010    0.4383    0.2578    0.1711    0.0252
    0.7244    0.6419    0.5793    0.3987    0.3121    0.1662
    0.8696    0.7872    0.7245    0.5440    0.4573    0.3114
    1.2231    1.1406    1.0780    0.8974    0.8108    0.6649
  Columns 7 through 10
    0.5834    0.7244    0.8696    1.2231
    0.5010    0.6419    0.7872    1.1406
    0.4383    0.5793    0.7245    1.0780
    0.2578    0.3987    0.5440    0.8974
    0.1711    0.3121    0.4573    0.8108
    0.0252    0.1662    0.3114    0.6649
         0    0.1410    0.2862    0.6397
    0.1410         0    0.1452    0.4987
    0.2862    0.1452         0    0.3535
    0.6397    0.4987    0.3535         0
diff =
    0.9718
ans =
  Columns 1 through 6
   -0.4558   -0.3795   -0.3215   -0.1544   -0.0742    0.0609
  Columns 7 through 10
    0.0842    0.2147    0.3492    0.6764
monproxpermut =
  Columns 1 through 6
         0    0.2612    0.2458    0.2612    0.2458    0.5116
    0.2612         0    0.2458    0.2458    0.4286    0.2458
    0.2458    0.2458         0    0.2458    0.5116    0.6899
    0.2612    0.2458    0.2458         0    0.2458    0.2458
    0.2458    0.4286    0.5116    0.2458         0    0.2612
    0.5116    0.2458    0.6899    0.2458    0.2612         0
    0.6080    0.5116    0.2458    0.2458    0.2458    0.2458
    0.6899    0.7264    0.2458    0.2612    0.5116    0.2458
    0.5116    0.5116    0.6899    0.6080    0.4286    0.2458
    1.2231    1.1406    1.0780    0.6899    0.7264    0.6080
  Columns 7 through 10
    0.6080    0.6899    0.5116    1.2231
    0.5116    0.7264    0.5116    1.1406
    0.2458    0.2458    0.6899    1.0780
    0.2458    0.2612    0.6080    0.6899
    0.2458    0.5116    0.4286    0.7264
    0.2458    0.2458    0.2458    0.6080
         0    0.1410    0.5116    0.6080
    0.1410         0    0.2458    0.4286
    0.5116    0.2458         0    0.2612
    0.6080    0.4286    0.2612         0
vaf =
    0.6672
\end{verbatim}

\large


\subsubsection{Some Concluding Comments}

The computational results presented in this paper suggest that for proximity matrices up to, say, $20 \times 20$, guaranteed optimal linear unidimensional scalings are easily done using just an interpreted M-file as long as one is working under Version 6.5 of MATLAB.  For matrices up to $25 \times 25$, the Fortran enhanced routine would provide the computational strategy of choice.  Beyond matrices of size 25, either  iterative QA  or Pliner's smoothing strategy should lead to optimal solutions (although not verifiably so) upon the use of some number of random starts.   Generally, both the original Guttman (1968) update strategy or the nonlinear programming reformulation of Lau, Lam, and Tse (1998) are problematic, and should be avoided.

\smallskip

With respect to broader computational issues, there are four observations that might be made:
(1) the optimization of ``loops'' incorporated in MATLAB Version 6.5 (and in contrast to Version 6.1) is performing extremely well, and may in fact allow the simple use of raw M-files for situations where before, a formal search might have been undertaken to find  more computationally efficient alternatives; (2) compiling the type of C code that is constructed by the MATLAB C/C++ Compiler seems to lead to particularly inefficient routines, even as compared to just using the raw M-files under the new MATLAB Version 6.5; for questions of execution speed, the MATLAB Compiler has very questionable utility.  The issue then arises as to whether it is the C language/compiler per se, or the initial code generation  that leads to these particularly inefficient routines; (3) the use of Fortran subroutines (and their respective gateways) to carry out computationally intensive subtasks, can be extremely advantageous; for example, we can now construct verifiably optimal linear unidimensional scalings for fairly large problems (up to $25 \times 25$) within reasonable computation times.

\smallskip

Based on what we have learned computationally in this paper, the next logical steps would be to extend the presented MATLAB routines to  multiple linear unidimensional scales, and thus, to implement city-block multidimensional scaling.  These generalizations could include the movement away from the reliance on just the $L_{2}$ norm (say, to the $L_{1}$ norm as well), and to the automatic incorporation of optimal (monotonic) transformations of the given proximity matrix.  Included in these extensions would be an option for an ``individual differences model'' in which the ``group space'' would consist solely of object orders along the given axes, and the ``private spaces'' defined by individually specified coordinates for each of the separate sources that are consistent with respect to these group-space orders.

\begin{thebibliography}{99}

\bibitem{} Bodewig, E. (1956). \emph{Matrix calculus}. Amsterdam: North-Holland.

\bibitem{} Defays, D. (1978).  A short note on a method of seriation. \emph{British Journal of Mathematical and  Statistical Psychology}, \emph{3}, 49--53.

\bibitem{} de Leeuw, J., \& Heiser, W. (1977).  Convergence of correction-matrix algorithms for multidimensional scaling.  In J. C. Lingoes, E. E. Roskam, \& I. Borg (Eds.), \emph{Geometric representations of relational data} (pp.\ 735--752). Ann Arbor, MI: Mathesis Press.

\bibitem{} Dykstra, R. L. (1983). An algorithm for restricted least squares regression.  \emph{Journal of the American Statistical Association}, \emph{78}, 837--842.

\bibitem{} Francis, R. L., \& White, J. A. (1974). \emph{Facility layout and location: An analytical approach}. Englewood Cliffs, NJ: Prentice-Hall.

\bibitem{} Garey, M. R., \& Johnson, D. S. (1979). \emph{Computers and intractability}. San Francisco: W. H. Freeman.

\bibitem{} Guttman, L. (1968). A general nonmetric technique for finding the smallest coordinate space for a configuration of points.  \emph{Psychometrika}, \emph{33}, 469--506.

\bibitem{} Hubert, L. J., \& Arabie, P. (1986). Unidimensional scaling and combinatorial optimization.  In J. de Leeuw, W. Heiser, J. Meulman, \& F. Critchely (Eds.), \emph{Multidimensional data analysis} (pp.\ 181--196). Leiden, The Netherlands: DSWO Press.

\bibitem{} Hubert, L. J., Arabie, P., \& Meulman, J. (2001). \emph{Combinatorial data analysis: Optimization by dynamic programming}. Philadelphia: SIAM.

\bibitem{} Hubert, L. J., \& Schultz, J. W. (1976). Quadratic assignment as a general data analysis strategy. \emph{British Journal of Mathematical and Statistical Psychology}, \emph{29}, 190--241.

\bibitem{} Kaczmarz, S. (1937). Angen\"{a}herte Aufl\"{o}sung von Systemen linearer Gleichungen. \emph{Bulletin of the Polish Academy of Sciences}, \emph{A35}, 355--357.

\bibitem{} Lawler, E. L. (1975).  The quadratic assignment problem: A brief review.  In R. Roy (Ed.), \emph{Combinatorial programming: Methods and applications} (pp.\ 351--360).  Dordrecht, The Netherlands: Reidel.



\bibitem{} Lau, Kin-nam, Leung, Pui Lam, \& Tse, Ka-kit (1998). A nonlinear programming approach to metric unidimensional scaling. \emph{Journal of Classification}, \emph{15}, 3--14.

\bibitem{} Pardalos, P. M., \& Wolkowicz, H. (Eds.). (1994). \emph{Quadratic assignment and related problems}.  DIMACS Series on Discrete Mathematics and Theoretical Computer Science.  Providence, RI: American Mathematical Society.


\bibitem{} Pliner, V. (1996). Metric unidimensional scaling and global optimization. \emph{Journal of Classification}, \emph{13}, 3--18.

\bibitem{} Shepard, R. N. (1974). Representation of structure in similarity data: Problems and prospects.  \emph{Psychometrika}, \emph{39}, 373--421.


\bibitem{} Shepard, R. N., Kilpatric, D. W., \& Cunningham, J. P. (1975).  The internal representation of numbers. \emph{Cognitive Psychology}, \emph{7}, 82--138.


\end{thebibliography}

\subsection{An Update on the Timings Given in Table 2}


The table given below provides an update on the type of timings
possible for newer versions of Fortran (Intel Fortran Version 9.1;
32-bit --- the column label ``uniscaldpf.m''); supposedly faster
processors (MacBook Pro running Windows XP under Bootcamp; Intel
Core Duo at 2.16 GHz; 2 GB Ram --- all three columns below); newer
releases of MATLAB (Version 2007a --- column labeled
``uniscaldp.m''); comparing the automatic conversion provided by
\verb+matlab2fmex+ (under the column labeled ``ms\_uniscaldp.m''):

\bigskip

\begin{tabular}{cccc} \hline
 & & & \\
matrix size  & uniscaldp.m & ms\_uniscaldp.m & uniscaldpf.m \\ [1ex]
\hline

 & & & \\
10 & .05 & .22 & .00 \\
11 & .06 & .22 &  .00 \\
12 & .13 & .23 &  .01 \\
13 & .29 & .25 &  .01 \\
14 & .63 & .31 &  .03 \\
15 & 1.39 & .45 &  .07 \\
16 & 3.03 & .75 &  .15 \\
17 & 6.54 & 1.41 & .33 \\
18 & 14.15 & 2.88  & .75 \\
19 & 30.49 & 6.17  & 1.75 \\
20 & 65.48 & 13.34  & 3.95 \\
21 & 140.03 & 29.16 &  8.79 \\
22 & 298.87 & 63.46 &  19.29\\
23 & 636.36 & 137.57 & 42.12 \\
24 & 1352.90 & 297.33 &  91.23 \\
25 & 2876.28 & ---       &  196.45 \\
[1ex] \hline



\end{tabular}

\bigskip

\noindent Several conclusions in relation to Table 2 are immediate:
(a) the ``hand'' coding we did for the Fortran \verb+mex+ files (in
comparison to using \verb+matlab2fmex+), leads to code that is more
than three times faster; (b) the new processor does indeed increase
the speed of execution for the raw M-file (between, say, 15 and
20\%); (c) the newer Intel Fortran does not necessarily lead to an
automatic increase in execution speed compared to Compaq Visual 6.6
running on the slower processor.  In fact, there is a slight
decrease here (which might go away or reverse if we ``tinker'' with
the various settings during compilation for Intel Fortran).

\newpage


\subsection{Multidimensional Scaling in the City-Block Metric: $L_{1}$- and $L_{2}$-Norm Optimization Methods Using MATLAB}

In a previous paper (Hubert, Arabie, and Meulman, 2002; hereafter, referred to as HAM2002), a comparison is made among several different optimization strategies for the linear unidimensional scaling (LUS) task in the $L_{2}$-norm, with all implementations carried out within a MATLAB computational environment.  The central LUS task involves arranging the $n$ objects in a set $S$ = $\{O_{1}, O_{2}, \ldots, O_{n}\}$ along a single dimension, defined by coordinates, $x_{1}, x_{2},  \ldots, x_{n}$, based on an $n \times n$ symmetric proximity matrix $\mathbf{P}$ = $\{p_{ij}\}$, whose nonnegative entries are given a dissimilarity interpretation ($p_{ij} = p_{ji}$ for $1 \le i,j \le n$; $p_{ii} = 0$ for $1 \le i \le n$). The $L_{2}$ criterion
\begin{equation}
\sum_{i < j} (p_{ij}  - |x_{j} - x_{i}|)^{2} ,
\end{equation}
is minimized by the choice of the coordinates.  The present paper can be considered a close companion to this earlier piece, with extensions now given to multidimensional scaling in the city-block metric for both the $L_{1}$ and $L_{2}$ norms.  The computational routines to be discussed and illustrated are again freely available as MATLAB M-files.  We also note that most of the references given in this earlier paper would also be relevant here as background material on the basic LUS task, but that review will not be repeated here.  Also, we will not discuss (in this paper) comparisons to other methods (or strategies) for multidimensional scaling in the city-block metric  --- for the development of some of these alternatives, see Brusco (2001), Brusco and Stahl (2005), Groenen, Heiser, and Meulman (1999), Hubert, Arabie, and Meulman (1997), and Hubert, Arabie, and Hesson-McInnis (1992).

\smallskip

In the extensions to city-block multidimensional scaling being pursued, a slight generalization to the basic unidimensional task that incorporates an additional additive constant will prove extremely convenient.  We emphasize the more general least-squares loss function of the form
\begin{equation}
 \sum_{i < j} (p_{ij} - \{|x_{j} - x_{i}| - c \})^{2} ,
\end{equation}
where $c$ is some constant to be estimated along with the coordinates, $x_{1}, \ldots, x_{n}$; also,  the least sum-of-absolute-deviations (the $L_{1}$ loss function) is considered as a variant for estimating $c$ and the coordinates:
\begin{equation}
 \sum_{i < j} | p_{ij} - \{|x_{j} - x_{i}| - c \} | .
\end{equation}
Later sections remove the restriction to fitting only a single unidimensional structure to a symmetric proximity matrix, and rely on the type of computational approaches that include the augmentation by estimated additive constants.  Based on these latter strategies, extensions are given to the use of multiple unidimensional structures through a procedure of successive residualization of the original proximity matrix (even though in this process, negative residuals are encountered and need to be fitted).   For example, the fitting of two LUS structures to a proximity matrix $\{p_{ij}\}$ could be rephrased as the minimization of an $L_{2}$ loss function generalizing (10) to the form
\begin{equation}
\sum_{i < j} (p_{ij} - [|x_{j1} - x_{i1}| - c_{1}] - [|x_{j2} - x_{i2}| - c_{2}])^{2} ,
\end{equation}
or by extending (11) to the use of an $L_{1}$ loss function:
\begin{equation}
\sum_{i < j} |p_{ij} - [|x_{j1} - x_{i1}| - c_{1}] - [|x_{j2} - x_{i2}| - c_{2}] | .
\end{equation}

\smallskip

The attempt to minimize (12) or (13) could proceed with the fitting of a single LUS structure to $\{p_{ij}\}$, $[|x_{j1} - x_{i1}| - c_{1}]$, and once obtained, fitting a second LUS structure, $[|x_{j2} - x_{i2}| - c_{2}]$, to the residual matrix,
$\{p_{ij} - [|x_{j1} - x_{i1}| - c_{1}]\}$.  The process would then cycle by repetitively fitting the residuals from the second linear structure by the first, and the residuals from the first linear structure by the second, until the sequence converges.  In any case, obvious extensions would also exist to (12) and (13) for the inclusion of more than two LUS structures.

\smallskip

The explicit inclusion of two constants, $c_{1}$ and $c_{2}$, in (12) and (13) rather than adding these two together and including a single additive constant $c$, deserves some additional introductory explanation.  As would be the case in fitting a single LUS structure using the loss functions in (10) and (11), two interpretations exist for the role of the additive constant $c$.  We could consider $\{|x_{j} -  x_{i}|\}$ to be fitted to the translated proximities $\{p_{ij} + c \}$, or alternatively, $\{|x_{j} -  x_{i}| - c \}$ to be fitted to the original proximities $\{p_{ij}\}$, where the constant $c$ becomes part of the actual model.  Although these two interpretations do not lead to any algorithmic differences in how we would proceed with minimizing the loss functions in (10) and (11), a consistent use of the second interpretation suggests that we frame extensions to the use of multiple LUS structures as we did in (12) and (13), where it is explicit that the constants $c_{1}$ and $c_{2}$ are part of the actual models to be fitted to the (untransformed) proximities $\{p_{ij}\}$.  Once $c_{1}$ and $c_{2}$ are obtained, they could be summed as $c = c_{1} + c_{2}$, and an interpretation made that we have attempted to fit a transformed set of proximities $\{p_{ij} + c \}$ by the sum $\{|x_{j1} - x_{i1}| + |x_{j2} - x_{i2}|\}$ (and in this latter case, a more usual terminology would be one of a two-dimensional scaling (MDS) based on the city-block distance function).  However, such a further interpretation is unnecessary and could lead to at least some small terminological confusion in further extensions that we might wish to pursue.  For instance, if some type of (optimal nonlinear) transformation, say $f(\cdot)$, of the proximities is also sought (e.g., a monotonic function of some form) in addition to fitting multiple LUS structures, and where $p_{ij}$ in (12) or (13) is replaced by $f(p_{ij})$, and $f(\cdot)$ is to be constructed, the first interpretation would require the use of a `doubly transformed' set of proximities $\{f(p_{ij}) + c \}$ to be fitted by the sum $\{|x_{j1} - x_{i1}| + |x_{j2} - x_{i2}|\}$.  In general, it seems best to avoid the need to incorporate the notion of a double transformation in this context, and instead merely consider the constants $c_{1}$ and $c_{2}$ to be part of the models being fitted to a transformed set of proximities $f(p_{ij})$.

\smallskip

Although $L_{1}$ norms are possible to use within both the unidimensional and multidimensional contexts, and we give MATLAB M-files to do so, our general conclusion after experimentation is that they might be best avoided because of their generally needed much greater computationally expensive implementation without any particularly clear advantage. Given this lack of any prominent reason to use $L_{1}$ as opposed to $L_{2}$, the extension to a nonmetric multidimensional scaling in the city-block metric will be limited to the $L_{2}$ context.






\subsubsection{The Incorporation of Additive Constants in LUS}

We present and illustrate a MATLAB M-function, \verb+linfitac.m+, that fits in $L_{2}$ a given single unidimensional scale (by providing the coordinates, $x_{1}, \ldots, x_{n}$) and the additive constant ($c$) for some fixed input object ordering along the continuum defined by a permutation $\rho^{(0)}$.  This parallels directly the M-function given in HAM2002 called \verb+linfit.m+, but now  with an included additive constant estimation.  The computational mechanisms implemented in \verb+linfitac.m+ are reviewed below.  Two additional MATLAB M-functions, \verb+linfitl1.m+ and \verb+linfitl1ac.m+, are presented that fit within $L_{1}$ a given single unidimensional scale defined by a fixed object ordering $\rho^{(0)}$ without and with an additional additive constant respectively.  The computational basis for these functions are also reviewed.  Finally, we present two M-functions, \verb+uniscallp.m+ and \verb+uniscallpac.m+, that rely on an iterative linear programming strategy to actually identify best-fitting object orderings within the $L_{1}$ norm.  For the $L_{2}$ context, a comparable general purpose method for finding an object order was made available in HAM2002 based on iterative quadratic assignment (the M-function, \verb+uniscalqa.m+).

\paragraph{The $L_{2}$ Fitting of a Single Unidimensional Scale (with an Additive Constant)}

Given a fixed object permutation, $\rho^{(0)}$, we denote the set of all $n \times n$ matrices that are additive translations of the off-diagonal entries in the reordered symmetric proximity matrix $\{p_{\rho^{(0)}(i) \rho^{(0)}(j)}\}$ by $\Delta_{\rho^{(0)}}$, and let $\Xi$ be the set of all $n \times n$ matrices that represent the interpoint distances between all pairs of $n$ coordinate locations along a line.  Explicitly,

$\Delta_{\rho^{(0)}} \equiv \{\{q_{ij}\}\}$, where $q_{ij} = p_{\rho^{(0)}(i) \rho^{(0)}(j)} + c$, for some constant $c$, $i \ne j; q_{ii} = 0, 1 \le i,j \le n $;

 $\Xi \equiv \{\{r_{ij}\}\}$, where $r_{ij} = |x_{j} - x_{i}|$ for some set of $n$ coordinates, $x_{1} \le \cdots \le x_{n}$; $\sum_{i} x_{i} = 0$.

\noindent Alternatively, we could define $\Xi$ through a set of linear inequality (for non-negativity restrictions) and equality constraints (to represent the additive nature of distances along a line  -- as we did in \verb+linfit.m+ in HAM2002).  In any case, both $\Delta_{\rho^{(0)}}$ and $\Xi$ are closed convex sets (in a Hilbert space), and thus, given any $n \times n$ symmetric matrix with a zero main diagonal, its projection onto either $\Delta_{\rho^{(0)}}$ or $\Xi$ exists, i.e., there is a (unique) member of $\Delta_{\rho^{(0)}}$ or $\Xi$ at a closest (Euclidean) distance to the given matrix (e.g., see Cheney and Goldstein, 1959).  Moreover, if a procedure of alternating projections onto $\Delta_{\rho^{(0)}}$ and $\Xi$ is carried out (where a given matrix is first projected onto one of the sets, and that result is then projected onto the second which result is in turn projected back onto the first, and so on), the process is convergent and generates members of $\Delta_{\rho^{(0)}}$ and $\Xi$ that are closest to each other (again, this last statement is justified in Cheney and Goldstein, 1959, Theorems 2 and 4).

\smallskip

Given any $n \times n$ symmetric matrix with a main diagonal of all zeros, which we denote arbitrarily as $\mathbf{U} = \{u_{ij}\}$, its projection onto $\Delta_{\rho^{(0)}}$ may be obtained by a simple formula for the sought constant $c$.  Explicitly, the minimum over $c$ of
\[ \sum_{i < j} (\{p_{\rho^{(0)}(i) \rho^{(0)}(j)}\} + c - u_{ij})^{2} ,\]
is obtained for
\[ \hat{c} = (2/n(n-1)) \sum_{i < j} (u_{ij} - p_{\rho^{(0)}(i) \rho^{(0)}(j)}), \]
and thus, this last value defines a constant translation of the proximities necessary to generate that member of $\Delta_{\rho^{(0)}}$ closest to $\mathbf{U} = \{u_{ij}\}$.  For the second necessary projection and given any $n \times n$ symmetric matrix (again with a main diagonal of all zeros), that we denote arbitrarily as $\mathbf{V} = \{v_{ij}\}$ (but which in our applications will generally have the form $v_{ij} = p_{\rho^{(0)}(i) \rho^{(0)}(j)} \ + \ c $ for $i \ne j$ and some constant $c$), its projection onto $\Xi$ is somewhat more involved and requires minimizing
\[ \sum_{i < j} (v_{ij} - r_{ij})^{2} , \]
over $r_{ij}$, where $\{r_{ij}\}$ is subject to the linear inequality nonnegativity constraints, and the linear equality constraints of representing distances along a line (of the set $\Xi$).  Although this is a (classic) quadratic programming problem for which a wide variety of optimization techniques has been published, we adopt (as we did in fitting a LUS without an additive constant in \verb+linfit.m+), the Dykstra-Kaczmarz iterative projection strategy.

\paragraph{The MATLAB Function linfitac.m}

As discussed above, the MATLAB M-function, \verb+linfitac.m+, fits a set of coordinates to a given proximity matrix based on some given input permutation, say, $\rho^{(0)}$, plus an additive constant, $c$. The usage syntax of
\begin{verbatim}
[fit vaf coord addcon] = linfitac(prox,inperm)
\end{verbatim}
is similar to that of \verb+linfit.m+ (of HAM2002) except for the inclusion (as output) of the additive constant \verb+ADDCON+, and the replacement of the least-squares criterion of \verb+DIFF+ by the variance-accounted-for (\verb+VAF+) given by the general formula
\[ \mbox{vaf} = 1 - \frac{\sum_{i < j} (p_{\rho^{(0)}(i) \rho^{(0)}(j)} + c - |x_{j} - x_{i}|)^{2}}{\sum_{i < j} (p_{ij} - \bar{p})^{2}} ,\]
where $\bar{p}$ is the mean of the proximity values being used.

\smallskip

To illustrate the invariance of \verb+VAF+ to the use of linear transformations of the proximity matrix (although \verb+COORD+ and \verb+ADDCON+ obviously will change depending on the transformation used), we fit the permutation found optimal in the HAM2002 companion paper to two different matrices: the original proximity matrix for \verb+number.dat+, and one standardized to mean zero and variance one.  The latter matrix is obtained with the utility  \verb+proxstd.m+, with usage explained in its M-file header comments.

\smallskip

In the recording below, semicolons are placed after the invocation of the M-functions to initially suppress the output; transposes(') are then used on the output vectors to conserve space by only using row (as opposed to column) vectors in the listing; also, to conserve space, blank lines are always deleted in any output given throughout the paper.  Note that for the two proximity matrices used, the vaf values are the same (.5612) but the coordinates and additive constants differ; a listing of the standardized proximity matrix is given in the output to explicitly show how negative proximities pose no problem for the fitting process that allows the incorporation of additive constants within the fitted model.

\bigskip

\tiny

\begin{verbatim}
>> load number.dat
>> inperm = [1 2 3 5 4 6 7 9 10 8];
>> [fit vaf coord addcon] = linfitac(number,inperm);
>> vaf
vaf =
    0.5612
>> coord'
ans =
  Columns 1 through 5
   -0.3790   -0.2085   -0.1064   -0.0565   -0.0257
  Columns 6 through 10
    0.0533    0.1061    0.1714    0.1888    0.2565
>> addcon
addcon =
   -0.3089
>> numberstan = proxstd(number,0.0)
numberstan =
  Columns 1 through 5
         0   -0.5919    0.2105    0.8258    0.7027
   -0.5919         0   -1.2663   -0.9611    0.5157
    0.2105   -1.2663         0   -0.9217   -2.3739
    0.8258   -0.9611   -0.9217         0   -0.6313
    0.7027    0.5157   -2.3739   -0.6313         0
    1.2934    0.2302    0.6387   -0.5525   -0.6510
    1.2147    1.0670   -0.5919   -1.1876   -0.7544
    1.8103    0.4369    1.2541    0.2498    0.9882
    1.3771    1.2294   -0.8577    1.2934   -1.4534
    1.5199    0.4123    1.3131   -1.3697    0.6978
  Columns 6 through 10
    1.2934    1.2147    1.8103    1.3771    1.5199
    0.2302    1.0670    0.4369    1.2294    0.4123
    0.6387   -0.5919    1.2541   -0.8577    1.3131
   -0.5525   -1.1876    0.2498    1.2934   -1.3697
   -0.6510   -0.7544    0.9882   -1.4534    0.6978
         0   -0.7150   -0.6953    0.6387    0.2498
   -0.7150         0   -0.6116   -0.9414   -1.2072
   -0.6953   -0.6116         0   -0.6953   -0.4049
    0.6387   -0.9414   -0.6953         0   -0.7347
    0.2498   -1.2072   -0.4049   -0.7347         0
>> [fit vaf coord addcon] = linfitac(numberstan,inperm);
>> vaf
vaf =
    0.5612
>> coord'
ans =
  Columns 1 through 5
   -1.8656   -1.0262   -0.5235   -0.2783   -0.1266
  Columns 6 through 10
    0.2624    0.5224    0.8435    0.9292    1.2626
>> addcon
addcon =
    1.1437
\end{verbatim}

\large


\paragraph{The $L_{1}$ Fitting of a Single Unidimensional Scale (with an Additive Constant)}

The linear unidimensional scaling task in the $L_{1}$ norm can be phrased as one of finding a set of coordinates, $x_{1}, \ldots, x_{n}$, such that the $L_{1}$ criterion,
\begin{equation}
 \sum_{i < j} |p_{ij} - (|x_{j} - x_{i}| - c)| \ ,
\end{equation}
is minimized, where we now immediately include the possibility of an additive constant in the model (in what follows, $c$ can just be set to 0 for the more elemental model without an additive constant).  As an alternative reformulation of the optimization task in (14) that will prove convenient as a point of departure in our development of computational routines (much as what we did within the $L_{2}$ norm), we subdivide (14) into the two separate problems of finding a set of $n$ numbers, $x_{1} \le \cdots \le x_{n}$, and a permutation on the first $n$ integers, $\rho(\cdot) \equiv \rho$, for which
\begin{equation}
 \sum_{i < j} |p_{\rho(i) \rho(j)} - ((x_{j} - x_{i}) - c)|
\end{equation}
is minimized.  Again, we can impose the additional constraint that $\sum_{i=1}^{n} x_{i} = 0$.

\smallskip

Assuming for now that the permutation $\rho$ is given, the task of finding $x_{1} \le \cdots \le x_{n}$ to minimize (15) is a linear programming problem.  Without loss of generality, we let $\rho$ be the identity permutation and first rewrite $ \sum_{i < j} |p_{ij} - (|x_{j} - x_{i}| - c)|$ as the loss criterion $\sum_{i < j} (z_{ij}^{+} + z_{ij}^{-})$, where
\[  z_{ij}^{+} = \frac{1}{2}\{|p_{ij} - (|x_{j} - x_{i}| - c)| - (p_{ij} - (|x_{j} - x_{i}| - c))\}; \]
\[  z_{ij}^{-} = \frac{1}{2}\{|p_{ij} - (|x_{j} - x_{i}| - c)| + (p_{ij} - (|x_{j} - x_{i}| - c))\}, \]
for $1 \le i < j \le n$.  The unknowns are $c$, $x_{1},\ldots,x_{n}$, and for $1 \le i < j \le n$, $z_{ij}^{+}$, $z_{ij}^{-}$, and $y_{ij}$ $( \equiv |x_{j} - x_{i}|)$.  The constraints of the linear program take the form:
\[ -z_{ij}^{+} + z_{ij}^{-} + y_{ij} - c = p_{ij}; \]
\[ -x_{j} + x_{i} + y_{ij} = 0; \]
\[ z_{ij}^{+} \ge 0, z_{ij}^{-} \ge 0, y_{ij} \ge 0, \]
for $1 \le i < j \le n$, and
\[ x_{1} + \cdots + x_{n} = 0. \]

\paragraph{The MATLAB Functions linfitl1.m and linfitl1ac.m}

Based on the linear programming reformulation just given for finding a set of ordered coordinates for a fixed object permutation, the M-functions, \verb+linfitl1.m+ and \verb+linfitl1ac.m+, where the latter includes an additive constant in the model and the former does not, serve to setup the relevant (constraint) matrices for the associated linear programming task; the actual linear programming optimization is carried out by invoking \verb+linprog.m+ from the MATLAB Optimization Toolbox.

\smallskip

The syntax for \verb+linfitl1.m+ is
\begin{verbatim}
[fit diff coord exitflag] = linfitl1(prox,inperm)
\end{verbatim}
where if we denote the given permutation as $\rho^{0}(\cdot)$ (\verb+INPERM+), we seek a set of coordinates $x_{1} \le \cdots \le x_{n}$ (\verb+COORD+) to minimize (at a value of \verb+DIFF+)
\[ \sum_{i < j} |p_{\rho^{0}(i) \rho^{0}(j)} - |x_{j} - x_{i}|| ;\]
\verb+FIT+ refers to the matrix $\{|x_{j} - x_{i}|\}$, and \verb+EXITFLAG+ describes the exit condition of the linear program optimization (greater than 0 for convergence; 0 denotes the maximum number of function evaluations or iterations was exceeded; less than 0 indicates a failure of convergence to a solution).  For using \verb+linfitl1ac.m+, the syntax is
\begin{verbatim}
 [fit dev coord addcon exitflag] = linfitl1ac(prox,inperm)
\end{verbatim}
Here, we minimize
\[ \sum_{i < j} |p_{\rho^{0}(i) \rho^{0}(j)} - (|x_{j} - x_{i}| - c)| \]
where $c$ is given by \verb+ADDCON+ and \verb+DEV+ refers to the deviance(-accounted-for) defined by the normalized $L_{1}$ loss value:
\[ \mbox{DEV} = 1 - \frac{\sum_{i < j} |p_{\rho^{0}(i) \rho^{0}(j)} - (|x_{j} - x_{i}| - c)|}{\sum_{i < j} |p_{ij} - p_{med}|}, \]
where $p_{med}$ is the median of the off-diagonal proximity values.

\smallskip

We illustrate the use of \verb+linfitl1.m+ and \verb+linfitl1ac.m+ on the \verb+number.dat+ proximity matrix using the identity permutation as the input object order.  To conserve space, the \verb+FIT+ matrices are not listed below.

\bigskip

\tiny

\begin{verbatim}
>> load number.dat
>> inperm = 1:10;
>> [fit diff coord exitflag] = linfitl1(number,inperm);
Optimization terminated successfully.
>> diff
diff =
    8.2120
>> coord'
ans =
  Columns 1 through 6
   -0.7260   -0.4780   -0.2205   -0.1320   -0.1046    0.0780
  Columns 7 through 10
    0.2005    0.4022    0.4022    0.5784
>> exitflag
exitflag =
     1
>> [fit dev coord addcon exitflag] = linfitl1ac(number,inperm);
Optimization terminated successfully.
>> dev
dev =
    0.4252
>> coord'
ans =
  Columns 1 through 6
   -0.3511   -0.1956   -0.1129   -0.0809   -0.0137    0.0466
  Columns 7 through 10
    0.0930    0.1643    0.2020    0.2482
>> addcon
addcon =
   -0.3458
>> exitflag
exitflag =
     1
\end{verbatim}

\large

\subsubsection{Iterative Linear Programming}

Given the availability of the two linear programming based M-functions (discussed in previously) for fitting given unidimensional scales defined by specific input object permutations, it is possible to embed these two routines in a search strategy for actually finding the (at least hopefully) best such permutations in the first place.  This embedding is analogous to adopting iterative quadratic  assignment in \verb+uniscalqa.m+ (as given in HAM2002) and attempting to locate good unidimensional scalings in the $L_{2}$ norm.  Here, we have an iterative use of linear programming in \verb+uniscallp.m+  and \verb+uniscallpac.m+ to identify the good unidimensional scales in the $L_{1}$ norm, without and with, respectively, an additive constant in the fitted model.  The usage syntax of both M-functions are as follows:
\begin{verbatim}
[outperm coord diff fit] = uniscallp(prox,inperm)
[outperm coord dev fit addcon] = uniscallpac(prox,inperm)
\end{verbatim}
Both M-functions begin with a given object ordering (\verb+INPERM+) and evaluate the effect of pairwise object interchanges on the current permutation carried forward to that point.  If an object interchange is identified that improves the $L_{1}$ loss value, that interchange is made and the changed permutation becomes the current one.  When no pairwise object interchange can reduce \verb+DIFF+ in \verb+uniscallp.m+, or increase \verb+DEV+ in \verb+uniscallpac.m+ over its current value, that ending permutation is provided as \verb+OUTPERM+ along with its coordinates (\verb+COORD+) and the matrix \verb+FIT+ (the absolute differences of the ordered coordinates).  In \verb+uniscallpac.m+, the additive constant (\verb+ADDCON+) is also given.

\smallskip

The numerical example that follows relies on \verb+number.dat+ to provide the proximity matrix, and initializes both the M-functions with the identity permutation (1:10).  From other random starts that we have tried in addition to this very rational starting permutation, the resulting scales we give below are (almost undoubtedly) $L_{1}$-norm optimal.  We might note that the (optimal) object orderings differ depending on whether or not an additive constant is included in the model.

\bigskip

\tiny

\begin{verbatim}
>> load number.dat
>> [outperm coord diff fit] = uniscallp(number,1:10);
>> outperm
outperm =
     1     2     3     5     4     9     7     6    10     8
>> coord'
ans =
  Columns 1 through 6
   -0.7129   -0.3762   -0.2451   -0.1861   -0.0302    0.1081
  Columns 7 through 10
    0.1838    0.2229    0.4798    0.5559
>> diff
diff =
    7.0430
>> [outperm coord dev fit addcon] = uniscallpac(number,1:10);
>> outperm
outperm =
     1     2     3     4     5     7     6     9    10     8
>> coord'
ans =
  Columns 1 through 6
   -0.3807   -0.1647   -0.1087   -0.0637    0.0143    0.0903
  Columns 7 through 10
    0.1113    0.1283    0.1573    0.2163
>> dev
dev =
    0.4479
>> addcon
addcon =
   -0.3120
\end{verbatim}

\bigskip

\large

Although the M-functions are provided to either fit or find the best unidimensional scalings in the $L_{1}$ norm, we do not suggest their routine use.  The finding of the best unidimensional scales in $L_{1}$ is extremely expensive computationally (given the use of the repetitive linear programming subtasks), and without any obvious advantage over $L_{2}$, it is not clear why the $L_{1}$ approach should be pursued.  This same set of conclusions exist as well for the $L_{1}$ finding and fitting of multidimensional unidimensional scales.  (We might also mention a possible issue with ill-conditioning for some uses of \verb+linfitl1.m+ and \verb+linfitl1ac.m+ if the simplex option is chosen [and not the default interior-point algorithm] for the optimization method implemented in \verb+linprog.m+ (from the Optimization Toolbox for MATLAB 6.5).  The end results appear generally to be fine, but the intermediate warnings in either the finding or fitting of these unidimensional scales within $L_{1}$ is disconcerting.  So the use of the default interior-point strategy is recommended whenever \verb+linprog.m+ is called.)

\subsubsection{The $L_{2}$ Finding and Fitting of Multiple Unidimensional Scales}

As reviewed in the Introduction, the fitting of multiple unidimensional structures will be done by (repetitive) successive residualization, along with a reliance on the M-function, \verb+linfitac.m+, to fit each separate unidimensional structure, including its associated additive constant.  The M-function, \verb+biscalqa.m+, is a two-(or bi-)dimensional scaling strategy for the $L_{2}$ loss function of (12).  It has the syntax
\begin{verbatim}
[outpermone outpermtwo coordone coordtwo fitone fittwo ...
addconone addcontwo vaf] = biscalqa(prox,...
targone,targtwo,inpermone,inpermtwo,kblock,nopt)
\end{verbatim}
where the variables are similar to \verb+linfitac.m+, but with a suffix of \verb+ONE+ or \verb+TWO+ to indicate which one of the two unidimensional structures is being referenced.   The new variable \verb+NOPT+ controls the confirmatory or exploratory fitting of the two unidimensional scales; a value of \verb+NOPT = 0+ will fit in a confirmatory manner the two scales indicated by \verb+INPERMONE+ and \verb+INPERMTWO+; if \verb+NOPT = 1+, iterative  quadratic assignment (QA) (as discussed in HAM2002) is used to locate the better permutations to fit.


\smallskip


In the example given below, the input \verb+PROX+ is the standardized (to a mean of zero and a standard deviation of one) $10 \times 10$ proximity matrix based on \verb+number.dat+ (referred to as \verb+STANNUMBER+); \verb+TARGONE+ and \verb+TARGTWO+ are identical $10 \times 10$ equally-spaced target matrices; \verb+INPERMONE+ and \verb+INPERMTWO+ are different random permutations of the first 10 integers; \verb+KBLOCK+ is set at 2 (for the iterative QA subfunctions).  In the output, \verb+OUTPERMONE+ and \verb+OUTPERMTWO+ refer to the object orders; \verb+COORDONE+ and \verb+COORDTWO+ give the coordinates; \verb+FITONE+ and \verb+FITTWO+ are based on the absolute coordinate differences for the two unidimensional structures; \verb+ADDCONONE+ and \verb+ADDCONTWO+ are the two associated additive constraints; and finally, \verb+VAF+ is the variance-accounted-for in \verb+PROX+ by the two-dimensional structure.

\bigskip

\tiny

\begin{verbatim}
>> load number.dat
>> stannumber = proxstd(number,0.0);
>> inpermone = randperm(10);
>> inpermtwo = randperm(10);
>> kblock = 2;
>> nopt = 1;
>> [prox10,targone,targcir] = ransymat(10);
>> targtwo = targone;
>> [outpermone outpermtwo coordone coordtwo fitone fittwo...
addconone addcontwo vaf] = biscalqa(stannumber,targone,...
targtwo,inpermone,inpermtwo,kblock,nopt);
>> outpermone
outpermone =
  Columns 1 through 10
     1     2     3     4     5     6     7     9     8   10
>> outpermtwo
outpermtwo =
  Columns 1 through 10
     5     9     3     1     7     4    10     2     8    6
>> coordone'
ans =
  Columns 1 through 5
   -2.1786   -1.2363   -0.5418   -0.2915    0.0009
  Columns 6 through 10
    0.0858    0.6805    1.0310    1.0310    1.4191
>> coordtwo'
ans =
  Columns 1 through 5
   -0.8791   -0.8791   -0.8791   -0.2629   -0.1151
  Columns 6 through 10
    0.2472    0.2472    0.3639    0.9885    1.1688
>> addconone
addconone =
    1.3137
>> addcontwo
addcontwo =
    0.8803
>> vaf
vaf =
    0.8243
\end{verbatim}

\large

\bigskip

Although we have used the proximity matrix in \verb+number.dat+ primarily as a convenient numerical example to illustrate the various M-functions provided in the appendix, the substantive interpretation (as noted in more detail in HAM2002) for this particular two-dimensional structure is rather remarkable and worth pointing out.  The first dimension reflects number magnitude perfectly (in its coordinate order) with two objects (the actual digits 7 8) at the same (tied) coordinate value.  The second axis reflects the structural characteristics perfectly, with the coordinates split into the odd and even numbers (the digits 4 8 2 0 6 in the first five positions; 3 9 1 7 5 in the second five); there is a grouping of 4 8 2 at the same coordinates (reflecting powers of 2); a grouping of 6 3 9 (reflecting multiples of three) and of 3 9 at the same coordinates (reflecting the powers of 3); the odd numbers 7 5 that are not powers of 3 are at the extreme two coordinates of this second dimension.

\smallskip

Although we will not explicitly illustrate its use here, a tridimensional M-function, \verb+triscalqa.m+, is available that is an obvious generalization of \verb+biscalqa.m+.  The pattern of programming that this shows could be used directly as a pattern for extensions beyond three unidimensional structures.

\subsubsection{The $L_{1}$ Finding and Fitting of Multiple Unidimensional Scales}

In analogy to the $L_{2}$ fitting of multiple unidimensional structures, the use of the $L_{1}$ norm can again be done by (repetitive) successive residualization, but now with a reliance on the M-function, \verb+linfitl1ac.m+, to fit each separate unidimensional structure with its additive constant.  The M-function, \verb+biscallp.m+, is a two-(or bi-)dimensional scaling strategy for the $L_{1}$ loss-function
\begin{equation}
\sum_{i < j} |p_{ij} - [|x_{j1} - x_{i1}| - c_{1}] - [|x_{j2} - x_{i2}| - c_{2}]| ,
\end{equation}
with syntax (and all variables) similar to \verb+biscalqa.m+, including a provision for the confirmatory fitting of two given input orders (by setting \verb+NOPT = 0+).

\smallskip

In the example given below, the two beginning permutations used were the ones identified with the $L_{2}$ norm.  Compared to the input permutations, there are no changes in the output permutations.  The final deviance-accounted-for (\verb+DEV+) of this solution is .6787.

\tiny

\bigskip

\begin{verbatim}
>>load number.dat
>>stannumber = proxstd(number,0.0);
>>nopt = 1;
>>inpermone = [1 2 3 4 5 6 7 9 8 10];
>>inpermtwo = [5 3 9 1 7 4 10 2 8 6];
>>[outpermone outpermtwo coordone coordtwo fitone fittwo ...
addconone addcontwo dev] = ...
biscallp(stannumber,inpermone,inpermtwo,nopt);
>> outpermone
outpermone =
     1     2     3     4     5     6     7     9     8    10
>> outpermtwo
outpermtwo =
     5     3     9     1     7     4    10     2     8     6
>> coordone'
ans =
  Columns 1 through 6
   -1.9927   -1.1377   -0.4443   -0.3343    0.0611    0.1170
  Columns 7 through 10
    0.4919    0.7986    1.1162    1.3243
>> coordtwo'
ans =
  Columns 1 through 6
   -0.9351   -0.9351   -0.8721   -0.1714   -0.0055    0.1462
  Columns 7 through 10
    0.1462    0.4965    0.8732    1.2571
>> addconone
addconone =
    1.2366
>> addcontwo
addcontwo =
    0.8781
>> dev
dev =
    0.6787
\end{verbatim}

\bigskip

\large

As noted earlier, in principle one can use the $L_{1}$ norm in city-block scaling, but given the increased computational expense of doing so with no apparent advantage over $L_{2}$, it is suggested that the use of $L_{1}$ generally be avoided in favor of $L_{2}$.

\subsubsection{Incorporating Monotonic Transformation of a Proximity Matrix in Fitting Multiple Unidimensional Scales: $L_{2}$ Nonmetric Multidimensional Scaling in the City-Block Metric}



As a direct extension of the M-function, \verb+biscalqa.m+, discussed in the last section,  \verb+bimonscalqa.m+ provides an optimal mono\-tonic transformation (by incorporating the use of \verb+proxmon.m+ discussed in HAM2002) of the original proximity matrix given as input in addition to the later's bidimensional scaling.  To prevent degeneracy, the sum-of-squares value for the initial input proximity matrix is maintained in the optimally transformed proximities; the overall strategy is iterative with termination again dependent on a change in the variance-accounted-for being less than 1.0e-005.  The usage syntax is almost identical to that of \verb+biscalqa.m+ except for the inclusion of the monotonically transformed proximity matrix \verb+MONPROX+ as an output matrix:
\begin{verbatim}
[ ... monprox] = bimonscalqa( ... )
\end{verbatim}
The ellipses indicate that the same items should be used as in \verb+biscalqa.m+.  If \verb+bimonscalqa+ would have been used in the numerical example of the previous section, the same results given would have been provided initially plus the results for the optimally transformed proximity matrix.  We give this additional output below, which shows that the incorporation of an optimal monotonic transformation provides an increase in the \verb+VAF+ from .8243 to .9362; the orderings on the two dimensions remain the same as well as the nice substantive explanation of the previous section.


\bigskip

\tiny

\begin{verbatim}
>> outpermone
outpermone =
     1     2     3     4     5     6     7     9     8    10
>> outpermtwo
outpermtwo =
     9     5     3     1     7     4    10     2     8     6
>> coordone'
ans =
  Columns 1 through 6
   -2.3514   -1.3290   -0.6409   -0.3565    0.0775    0.1216
  Columns 7 through 10
    0.5857    1.1342    1.1342    1.6247
>> coordtwo'
ans =
  Columns 1 through 6
   -0.7793   -0.7793   -0.7793   -0.3891   -0.1196    0.3242
  Columns 7 through 10
    0.3242    0.3480    0.8467    1.0035
>> addconone
addconone =
    1.4394
>> addcontwo
addcontwo =
    0.7922
>> vaf
vaf =
    0.9362
>> monprox
monprox =
  Columns 1 through 6
         0   -0.7387   -0.1667    0.5067    0.5067    1.4791
   -0.7387         0   -0.8218   -0.8218    0.5067   -0.1667
   -0.1667   -0.8218         0   -0.8218   -1.6174    0.5067
    0.5067   -0.8218   -0.8218         0   -0.7387   -0.7387
    0.5067    0.5067   -1.6174   -0.7387         0   -0.7387
    1.4791   -0.1667    0.5067   -0.7387   -0.7387         0
    1.0321    0.5067   -0.7387   -0.8218   -0.8218   -0.8218
    2.6590    0.5067    1.0321   -0.1667    0.5067   -0.8218
    1.7609    1.0321   -0.8218    1.0321   -1.2541    0.5067
    2.6231    0.5067    1.4791   -0.8218    0.5067   -0.0534
  Columns 7 through 10
    1.0321    2.6590    1.7609    2.6231
    0.5067    0.5067    1.0321    0.5067
   -0.7387    1.0321   -0.8218    1.4791
   -0.8218   -0.1667    1.0321   -0.8218
   -0.8218    0.5067   -1.2541    0.5067
   -0.8218   -0.8218    0.5067   -0.0534
         0   -0.7387   -0.8218   -0.8218
   -0.7387         0   -0.7387   -0.7387
   -0.8218   -0.7387         0   -0.8218
   -0.8218   -0.7387   -0.8218         0
\end{verbatim}

\bigskip

\large


Although we will not provide an example of its use here, \verb+trimonscalqa.m+  extends \verb+triscalqa.m+  to include an optimal monotonic  transformation of whatever is given as the original input proximity matrix.


\subsubsection{Concluding Comments}

An obvious conclusion to this section is that if one is interested in (nonmetric) city-block scaling in two or three dimensions within $L_{2}$, the MATLAB routines, \verb+biscalqa.m+ and \verb+bimonscalqa.m+; \verb+triscalqa.m+ and \verb+trimonscalqa.m+, would be natural alternatives to consider.  The $L_{1}$ metric is not a computationally competitive alternative, and appears to be without any particular discernible advantage to make up for its greater expense.  In any case, one aspect of all of these given M-files that we have not emphasized but will in these concluding comments, is their possible usage in the confirmatory context (by setting the \verb+NOPT+ switch to \verb+0+), and fitting various fixed object orderings in multiple dimensions.  One possible application of this type of confirmatory fitting would be in an individual differences scaling context.  Explicitly, we begin with a collection of, say, $K$ proximity matrices, $\mathbf{P}_{1}, \ldots, \mathbf{P}_{K}$, obtained from $K$ separate sources, and through some weighting and averaging process construct a single aggregate proximity matrix, $\mathbf{P}_{A}$. On the basis of $\mathbf{P}_{A}$, suppose a  two-dimensional city-block scaling is constructed (using, say, \verb+biscalqa.m+); we label the latter the ``common space'' in consistency with what is usually done in the weighted Euclidean model (e.g., see the INDSCAL model of Carroll and Chang, 1970, or the PROXSCAL program in the Categories Module of SPSS --- Busing, Commandeur, and Heiser, 1997).  Each of the $K$ proximity matrices then can be used in a confirmatory fitting of the object orders along the two axes.  Thus, a very general ``private space'' is generated for each source and where the actual coordinates along both axes are unique to that source, subject only to the object order constraints of the group space.  This strategy provides an individual differences model generalization over the usual weighted Euclidean model where the latter allows only differential axes scaling (stretching or shrinking) in generating the private spaces.


\begin{thebibliography}{99}

\bibitem{} Brusco, M. J. (2001).  A simulated annealing heuristic for unidimensional and multidimensional (city-block) scaling of symmetric proximity matrices.  \emph{Journal of Classification}, \emph{18}, 3--33.

\bibitem{} Brusco, M. J., \& Stahl, S. (2005).  Optimal least-squares unidimensional scaling: Improved branch-and-bound procedures and comparison to dynamic programming. \emph{Psychometrika}, \emph{70}, 253--270..


\bibitem{} Busing, F. M. T. A., Commandeur, J. J. F., \& Heiser, W. J. (1997).  PROXSCAL: A multidimensional scaling program for individual differences scaling with constraints.  In W. Bandilla \& F. Faulbaum (Eds.), \emph{Softstat '97: Advances in Statistical Software, Volume 6} (pp.\ 67--74).  Stuttgart: Lucius \& Lucius.

\bibitem{} Carroll, J. D., \& Chang, J. J. (1970). Analysis of individual differences in multidimensional scaling via an N-way generalization of Eckart-Young decomposition.  \emph{Psychometrika}, \emph{35}, 283--319.

\bibitem{} Cheney, W., \& Goldstein, A. (1959).  Proximity maps for convex sets. \emph{Proceedings of the American Mathematical Society}, \emph{10}, 448--450.

\bibitem{} Groenen, P. J. F., Heiser, W. J., \& Meulman, J. J. (1999).  Global optimization in least-squares multidimensional scaling by distance smoothing.  \emph{Journal of Classification}, \emph{16}, 225--254.

\bibitem{} Hubert, L. J., Arabie, R., \& Hesson-McInnis, M. (1992).  Multidimensional scaling in the city-block metric: A combinatorial approach.  \emph{Journal of Classification}, \emph{9}, 211--236.

\bibitem{} Hubert, L. J., Arabie, R., \& Meulman, J. J. (1997).  Linear and circular unidimensional scaling for symmetric proximity matrices.  \emph{British Journal of Mathematical and Statitical Psychology}, \emph{50}, 253--284.


\bibitem{} Hubert, L. J., Arabie, R., \& Meulman, J. J. (2002).  Linear unidimensional scaling in the $L_{2}$-norm: Basic optimization methods using MATLAB.  \emph{Journal of Classification}, \emph{19}, 303--328.

\end{thebibliography}

\newpage

\section{Confirmatory Assignment Methods}

The confirmatory assignment methods discussed in this section fall into four categories: linear, quadratic, cubic, and quartic.  All of these rely on the generation of permutation distributions (either exact, through complete enumeration; or approximate,  through random sampling), plus the subsequent comparison of some observed index to these distributions.  We present each of the strategies below in order.

\subsection{Linear Assignment}

The Linear Assignment (LA) model can be characterized through an $n \times n$ matrix (square but not necessarily symmetric), $\mathbf{A} = \{a_{ij}\}$, that gives ``values'' pertaining to the match of particular rows to particular columns.  If a permutation of the first $n$ integers, $\rho(\cdot)$, specifies a specific row/column matching (i.e., row $i$ is matched to column $\rho(i)$), then the attendant index for the permutation is
\[ \mathcal{L}(\rho) = \sum_{i=1}^{n} a_{i\rho(i)} \ . \]
\noindent Usually, the raw index that we wish to evaluate is based on the identity permutation, i.e., the trace of $\mathbf{A}$, $\sum_{i=1}^{n} a_{ii}$.  We assume this is the default case from now on; thus, the tr($\mathbf{A}$) is compared to the distribution of $\mathcal{L}(\rho)$, where all $n!$ permutations are considered equally-likely.

\smallskip

The primary M-file for carrying out the LA comparison is \verb+linassign.m+, with syntax
\begin{verbatim}
[rawindex,mean,variance,upvalue,sortvalues] = ...
     linassign(prox,nstarts)
\end{verbatim}
\noindent Here, the salience of the sum of main diagonal entries (the trace) in the $n \times n$ square (but not necessarily symmetric) input matrix \verb+PROX+ is evaluated using a linear assignment permutation strategy.  The columns of \verb+PROX+ are randomly permuted \verb+NSTARTS+ times with the matrix trace repeatedly computed.  The original trace of \verb+PROX+ is \verb+RAWINDEX+; the \verb+NSTARTS+ index values are given (sorted in increasing order) in the vector \verb+SORTVALUES+; \verb+MEAN+ and \verb+VARIANCE+ are the exact mean and variance of the permutation distribution assuming that all $n!$ permutations are equally-likely; \verb+UPVALUE+ is the proportion of the \verb+NSTARTS+ indices that are as large or larger than \verb+RAWINDEX+ (i.e., \verb+UPVALUE+ is the upper-tail $p$-value for the observed index, \verb+RAWINDEX+).

\smallskip

A second M-file called \verb+linassign_exact.m+ carries out the same LA comparison but uses complete enumeration of the $n!$ possible permutations.  The syntax is the same as \verb+linassign.m+  except there is no \verb+NSTARTS+ input number.  Given the computational storage required for complete enumeration, \verb+linassign_exact.m+ should probably be restricted to $n$'s less than, say, 9.

\smallskip

The closed-form formulas are given below for the mean and variance of $\mathcal{L}(\rho)$ over all $n!$ equally-likely permutations:

\[ \mathrm{E}(\mathcal{L}(\rho)) = \frac{1}{n} \sum_{i,j} a_{ij} \ ; \]

\[ \mathrm{V}(\mathcal{L}(\rho)) = \frac{1}{n(n-1)} \{ (\frac{1}{n} (\sum_{i,j} a_{ij})^{2} - \sum_{j=1}^{n} (\sum_{i=1}^{n} a_{ij})^{2} - \sum_{i=1}^{n} (\sum_{j=1}^{n} a_{ij})^{2} + n \sum_{i,j} a_{ij}^{2} \} \ . \]

\subsection{Quadratic Assignment}

The Quadratic Assignment (QA) model can be characterized through two $n \times n$ matrices, $\mathbf{A} = \{a_{ij}\}$ and $\mathbf{B} = \{b_{ij}\}$ (square, but not necessarily symmetric; main diagonals of zero are assumed for both).  If $\rho(\cdot)$ specifies a simultaneous permutation of the rows and columns of $\mathbf{A}$, then the relevant index for the permutation is
\[ \mathcal{Q}(\rho) = \sum_{i,j} a_{\rho(i) \rho(j)} b_{ij} \ . \]
\noindent Again, the index for the identity permutation, $\sum_{i,j} a_{ij} b_{ij}$, is compared to the distribution of $\mathcal{Q}(\rho)$, with the $n!$ permutations again considered equally-likely.

\smallskip

The primary M-file for the QA comparison task is \verb+quadassign.m+ with syntax:
\begin{verbatim}
[rawindex,expectation,variance,upvalue,sortvalues] = ...
     quadassign(proxa,proxb,nstarts)
\end{verbatim}
\noindent Here, two $n \times n$ square (but not necessarily symmetric) input matrices, \verb+PROXA+ and \verb+PROXB+ (both with main diagonals of zero) are compared using a (quadratic assignment) permutation strategy.  The rows and columns of \verb+PROXA+ are randomly permuted \verb+NSTARTS+ times with the cross-product index  repeatedly computed with respect to \verb+PROXB+.  The raw cross-product index between \verb+PROXA+ and \verb+PROXB+ is \verb+RAWINDEX+; the \verb+NSTARTS+ cross-product index values are given (sorted in increasing order) in the vector \verb+SORTVALUES+; \verb+EXPECTATION+ and \verb+VARIANCE+ are the exact mean and variance of the permutation distribution, assuming that all $n!$ permutations are equally-likely; \verb+UPVALUE+ is the proportion of the \verb+NSTARTS+ indices that are as large or larger than \verb+RAWINDEX+ (i.e., \verb+UPVALUE+ is the upper-tail $p$-value for the observed index \verb+RAWINDEX+).  As in the LA comparison M-files, a second complete enumeration routine, \verb+quadassign_exact+, is also available.

\smallskip

Closed-form formulas that are used in the QA M-files for the mean and variance of $\mathcal{L}(\rho)$, are given below:

\[ \mathrm{E}(\mathcal{L}(\rho)) = \frac{1}{n(n-1)} \sum_{i,j} a_{ij} \sum_{i,j} b_{ij} \ ; \]

\[ \mathrm{V}(\mathcal{L}(\rho))  = \frac{-B_{1}}{n^{2}(n-1)^{2}}   + \frac{B_{2} + B_{3}}{n(n-1)}  + \frac{B_{4} + 2B_{5} + B_{6}}{n(n-1)(n-2)} + \frac{B_{7}}{n(n-1)(n-2)(n-3)} \ ,  \]

\noindent for:

\[ B_{1} = (\sum_{i,j} a_{ij})^{2} (\sum_{i,j} b_{ij})^{2} \ ; \]

\[ B_{2} = (\sum_{i,j} a_{ij}^{2}) (\sum_{i,j} b_{ij}^{2}) \ ; \]

\[ B_{3} = (\sum_{i,j} a_{ij}a_{ji}) (\sum_{i,j} b_{ij}b_{ji}) \ ; \]

\[ B_{4} = \sum_{i=1}^{n}  (\sum_{j=1}^{n} a_{ij})^{2} - \sum_{i,j} a_{ij}^{2} )\sum_{i=1}^{n}  (\sum_{j=1}^{n} b_{ij})^{2} - \sum_{i,j} b_{ij}^{2} ) \ ; \]

\[ B_{5} = (\sum_{j=1}^{n} (\sum_{i=1}^{n} a_{ij})(\sum_{i=1}^{n} a_{ji}) - \sum_{i,j} a_{ij} a_{ji})(\sum_{j=1}^{n} (\sum_{i=1}^{n} b_{ij})(\sum_{i=1}^{n} b_{ji}) - \sum_{i,j}  b_{ij} b_{ji}) \ ; \]

\[ B_{6} = \sum_{j=1}^{n} ((\sum_{i=1}^{n} a_{ij})^{2} - \sum_{i,j} a_{ij}^{2} ) \sum_{j=1}^{n} ((\sum_{i=1}^{n} b_{ij})^{2} - \sum_{i,j} b_{ij}^{2} ) \ ;  \]

\[ B_{7} = ( (\sum_{i,j} a_{ij})^{2} - \sum_{i=1}^{n} (\sum_{j=1}^{n} a_{ij})^{2} -2 \sum_{j=1}^{n}(\sum_{i=1}^{n} a_{ij})(\sum_{i=1}^{n} a_{ji}) -\sum_{j=1}^{n}(\sum_{i=1}^{n} a_{ij})^{2} + \sum_{i,j} a_{ij}a_{ji} + \sum_{i,j} a_{ij}^{2}) \times \]

\[ ( (\sum_{i,j} b_{ij})^{2} - \sum_{i=1}^{n} (\sum_{j=1}^{n} b_{ij})^{2} -2 \sum_{j=1}^{n}(\sum_{i=1}^{n} b_{ij})(\sum_{i=1}^{n} b_{ji}) -\sum_{j=1}^{n}(\sum_{i=1}^{n} b_{ij})^{2} + \sum_{i,j} b_{ij}b_{ji} + \sum_{i,j} b_{ij}^{2}) \ . \]

\subsection{Cubic and Quartic Assignment}

The M-files implementing cubic and quartic assignment to compare two matrices, $\mathbf{A}$ and $\mathbf{B}$, have identical syntaxes.  For cubic assignment, which relies on the product of three-place comparison functions, we have the form
\begin{verbatim}
[rawindex,sampmean,sampvariance,upvalue,ncompare,sortvalues] = ...
     cubeassign(proxa,proxb,nstarts)
\end{verbatim}
\noindent Just as for \verb+quadassign.m+, we compare the two $n \times n$ square (but not necessarily symmetric) input matrices \verb+PROXA+ and \verb+PROXB+ (both with main diagonals of zero) using a (cubic assignment) permutation strategy.  The rows and columns of \verb+PROXA+ are randomly permuted \verb+NSTARTS+ times with the triad cross-product index (twice the number of consistently ordered row triads minus the inconsistent triads) between \verb+PROXA+ and  \verb+PROXB+ is \verb+RAWINDEX+; this is given explicitly by
\[ \sum_{\mathrm{distinct} \ i,j,k} \mathrm{sign}(a_{ik} - a_{ij})  \mathrm{sign}(b_{ik} - b_{ij}) \]

\noindent where

\[ \mathrm{sign}(x) = +1, 0, -1 \ \mathrm{for} \  x > 0, x = 0, x < 0 \ . \]

\noindent \verb+NCOMPARE+ is the maximum that  \verb+RAWINDEX+ could be, while taking into account the tied proximity values in \verb+PROXA+ and \verb+PROXB+; the \verb+NSTARTS+ cross-product index values are given (sorted in increasing order) in the vector \verb+SORTVALUES+; \verb+UPVALUE+ is the proportion of the \verb+NSTARTS+ indices that are as large or larger than \verb+RAWINDEX+ (i.e., \verb+UPVALUE+ is the upper-tail $p$-value for the observed index \verb+RAWINDEX+).

\smallskip

The use of quartic assignment and the M-file, \verb+quartassign.m+, follows that of \verb+cubicassign.m+, except that the comparison function is  now a four-place cross-product index (defined as twice the number of consistent quadruples minus the inconsistent quadruples) between \verb+PROXA+ and \verb+PROXB+:

\[ \sum_{i \ne j, k \ne l} \mathrm{sign}(a_{kl} - a_{ij})  \mathrm{sign}(b_{kl} - b_{ij}) \ . \]

\subsubsection{Some Useful Matrices for Confirmatory Comparisons}

This section summarizes eight M-files that provide ready-made matrices helpful in a number of confirmatory comparisons.  The help comments are given for each along with a small numerical example (of size $6 \times 6$:

\small

\begin{verbatim}
>> help partition_pairweighted

  PARTITION_PAIRWEIGHTED produces a symmetric paroximty matrix of size $n
  \times n$ (with zero main diagonal) that contains K on-diagonal blocks
  each of size $n_{k} \times n_{k}$ for $k = 1,...,K (the sum of the class
  sizes is $n$), with values (to produce average within-class proximities)
  of $1/n_{k}*(n_{k}-1)$; all other matrix entries are zero.  CLASS_SIZES
  is a $K \times 1$ vector containing the class class sizes $n_{1},...,n_{K}$

  syntax:  [partitionpairweighted] = partition_pairweighted(class_sizes)

>> [partitionpairweighted] = partition_pairweighted([2 4])

partitionpairweighted =

         0    0.5000         0         0         0         0
    0.5000         0         0         0         0         0
         0         0         0    0.0833    0.0833    0.0833
         0         0    0.0833         0    0.0833    0.0833
         0         0    0.0833    0.0833         0    0.0833
         0         0    0.0833    0.0833    0.0833         0

>> help path_linear

  PATH_LINEAR produces an asymmetric proximity matrix of size $n \times n$
  containing all zeros except for ones placed immediately about the
  main-diagonal.  This produces an ordered
  linear path from 1 to 2 to ... to n.

  syntax: [pathlinear] = path_linear(n)

>> [pathlinear] = path_linear(6)

pathlinear =

     0     1     0     0     0     0
     0     0     1     0     0     0
     0     0     0     1     0     0
     0     0     0     0     1     0
     0     0     0     0     0     1
     0     0     0     0     0     0

>> help path_circle

  PATH_CIRCLE produces an asymmetric proximity matrix of size $n \times n$
  containing all zeros except for ones placed immediately about the
  main-diagonal and at the (1,n) position.  This produces an ordered
  circular path from 1 to 2 to ... to n and back to 1.

  syntax: [pathcircular] = path_circle(n)

>> [pathcircular] = path_circle(6)

pathcircular =

     0     1     0     0     0     1
     0     0     1     0     0     0
     0     0     0     1     0     0
     0     0     0     0     1     0
     0     0     0     0     0     1
     0     0     0     0     0     0

>> help subset_aver_within

  SUBSET_AVER_WITHIN produces a symmetric proximity matrix of size $n
  \times n$ that contains all zeros except for the $k \times k$
  upper-left-hand block (those off-diagonal entries are $1/(k*(k-1))$, to
  produce an average subset proximity value in a second matrix).

  syntax: [subsetaveragewithin] = subset_aver_within(n,k)

>> [subsetaveragewithin] = subset_aver_within(6,4)

subsetaveragewithin =

         0    0.0833    0.0833    0.0833         0         0
    0.0833         0    0.0833    0.0833         0         0
    0.0833    0.0833         0    0.0833         0         0
    0.0833    0.0833    0.0833         0         0         0
         0         0         0         0         0         0
         0         0         0         0         0         0

>> help subset_aver_within_versus_between

  SUBSET_AVER_WITHIN_VERSUS_BETWEEN produces a symmetric proximity matrix of size $n
  \times n$ that contains all zeros except for the $k \times k$
  upper-left-hand block (those off-diagonal entries are $1/(k*(k-1))$, to
  produce an average subset proximity value in a second matrix), and the
  upper-left-hand ($k \times (n-k)$) and lower-right-hand ($(n-k) \times k$) corners
  (these values are $-1/(2*k*(n-k))$ to "take away" a between subset average).

  syntax: [subsetaveragewithin_between] = subset_aver_within_versus_between(n,k)

>> [subsetaveragewithin_between] = subset_aver_within_versus_between(6,4)

subsetaveragewithin_between =

         0    0.0833    0.0833    0.0833   -0.0625   -0.0625
    0.0833         0    0.0833    0.0833   -0.0625   -0.0625
    0.0833    0.0833         0    0.0833   -0.0625   -0.0625
    0.0833    0.0833    0.0833         0   -0.0625   -0.0625
   -0.0625   -0.0625   -0.0625   -0.0625         0         0
   -0.0625   -0.0625   -0.0625   -0.0625         0         0

>> help targ_flow

  TARG_FLOW produces an asymmetric proximity matrix of size $n \times n$
  containing zeros below the main diagonal and ones above the main
  diagonal.  When compared (through a cross-product) against another matrix,
  the sum of the above-main-diagonal entries is taken.

  syntax: [targflow] = targ_flow(n)

>> [targflow] = targ_flow(6)

targflow =

     0     1     1     1     1     1
     0     0     1     1     1     1
     0     0     0     1     1     1
     0     0     0     0     1     1
     0     0     0     0     0     1
     0     0     0     0     0     0

>> help targcir

   TARGCIR produces a symmetric proximity matrix of size
   $n \times n$, containing distances
   between equally and unit-spaced positions
   around a circle: targcircular(i,j) = min(abs(i-j),n-abs(i-j)).

   syntax: [targcircular] = targcir(n)

>> [targcircular] = targcir(6)

targcircular =

     0     1     2     3     2     1
     1     0     1     2     3     2
     2     1     0     1     2     3
     3     2     1     0     1     2
     2     3     2     1     0     1
     1     2     3     2     1     0

>> help targlin

   TARGLIN produces a symmetric proximity matrix of size
   $n \times n$, containing distances
   between equally and unit-spaced positions
   along a line: targlinear(i,j) = abs(i-j).

   syntax: [targlinear] = targlin(n)

>> [targlinear] = targlin(6)

targlinear =

     0     1     2     3     4     5
     1     0     1     2     3     4
     2     1     0     1     2     3
     3     2     1     0     1     2
     4     3     2     1     0     1
     5     4     3     2     1     0
\end{verbatim}

\large






\section{Concave and Convex Monotonic (Isotonic) Regression}

This section gives four M-files that implement extensions to the monotonic regression task by imposing additional convexity and/or concavity restrictions on the regression function.  The basic optimization problem involves two $n \times 1$ vectors, say $\mathbf{x} = \{x_{i}\}$ (considered as an independent variable) and  $\mathbf{y} = \{y_{i}\}$ (considered as a dependent variable).  We wish to find an $n \times 1$ vector  $\hat{\mathbf{y}} = \{\hat{y}_{i}\}$, such that the least-squares criterion,
\[ \sum_{i=1}^{n} (y_{i} - \hat{y}_{i})^{2} \ , \]
\noindent is minimized, subject to $\hat{\mathbf{y}}$ satisfying some particular set of constraints.  For convenience, and without-loss-of-any-generality, assume the entries in $\mathbf{x}$ are indexed,  so: $x_{i} \le x_{i+1}$ for $i = 1,\ldots,n-1$.  (It is easier to specify the type of additional constraints we wish to impose if this initial ordering is assumed.  In the various M-files introduced below, the input arguments are $\mathbf{x}$ and $\mathbf{y}$, but no preliminary ordering of the values in $\mathbf{x}$ is needed (or assumed).)


\paragraph{Monotonic Regression}

(A) if $x_{i} < x_{i+1}$, for $i = 1,\ldots,n-1$, then $\hat{y}_{i} \le \hat{y}_{i+1}$;

\paragraph{Convex Monotonic Regression}

(B) in addition to Condition (A), if $x_{i-1} < x_{i} < x_{i+1}$, for $i = 2,\ldots,n-1$,

\[ \hat{y}_{i}  \le  ( \frac{x_{i+1} - x_{i}}{x_{i+1} - x_{i-1}}) \hat{y}_{i-1} + ( \frac{x_{i} - x_{i-1}}{x_{i+1} - x_{i-1}}) \hat{y}_{i+1} \ . \]

\noindent Condition (B) implies that the fitted value, $\hat{y}_{i}$, associated with $x_{i}$, lies at or below the line segment connecting $(x_{i-1},\hat{y}_{i-1})$  and $(x_{i+1},\hat{y}_{i+1})$.  Or, equivalently, the slope of the line segment connecting $(x_{i-1},\hat{y}_{i-1})$ and $(x_{i},\hat{y}_{i})$ is no greater than that connecting $(x_{i},\hat{y}_{i})$ and $(x_{i+1},\hat{y}_{i+1})$.

\paragraph{Concave Monotonic Regression}

(C) in addition to Condition (A), if $x_{i-1} < x_{i} < x_{i+1}$, for $i = 2,\ldots,n-1$,

\[ \hat{y}_{i}  \ge  ( \frac{x_{i+1} - x_{i}}{x_{i+1} - x_{i-1}}) \hat{y}_{i-1} + ( \frac{x_{i} - x_{i-1}}{x_{i+1} - x_{i-1}}) \hat{y}_{i+1} \ . \]

\noindent As can be seen, Condition (C) merely changes the sense of the inequality in (B), and implies that the fitted value, $\hat{y}_{i}$, associated with $x_{i}$, lies at or above the line segment connecting $(x_{i-1},\hat{y}_{i-1})$  and $(x_{i+1},\hat{y}_{i+1})$.  Or, equivalently, the slope of the line segment connecting $(x_{i-1},\hat{y}_{i-1})$ and $(x_{i},\hat{y}_{i})$ is no less than that connecting $(x_{i},\hat{y}_{i})$ and $(x_{i+1},\hat{y}_{i+1})$.

\paragraph{Convex-Concave Monotonic Regression}

To define Condition (D), a monotonic convex function is fit up to the median object (based on the assumed ordered objects in $\mathbf{x}$) and a concave monotonic function thereafter (so, an ``ogive-like'' shape is given).  Explicitly, an \verb+upper_limit+ is defined on the object triples considered in fitting convexity, and a \verb+lower_limit+ on the triples considered in fitting concavity.  For $n$ odd, \verb+upper_limit+ = $(n+3)/2$; \verb+lower_limit+ =  $(n-1)/2$; for $n$ even,  \verb+upper_limit+ = $(n/2) + 1$; \verb+lower_limit+ =  $n/2$.

\smallskip

The four M-files implementing these four fitting constraints have more or less the same syntax, and are named:

\bigskip

Condition (A): \verb+monotonic_regression_dykstra.m+

Condition (B): \verb+convex_monotonic_regression_dykstra.m+

Condition (C): \verb+concave_monotonic_regression_dykstra.m+

Condition (D): \verb+convex_concave_monotonic_regression_dykstra.m+

\bigskip

\noindent For example, syntax for \verb+convex_concave_monotonic_regression_dykstra.m+ is
\begin{verbatim}
[yhat,vaf_yhat] = convex_concave_monotonic_regression_dykstra(x,y)
\end{verbatim}
\noindent with the vector of values in \verb+YHAT+ weakly monotonic with respect to the values in the independent input vector \verb+X+ (which is \emph{not} necessarily ordered); defining a convex function up to the median observation in \verb+X+ and a concave function thereafter; and which minimizes the least-squares loss measure, $\sum_{i} (y_{i} - \hat{y}_{i})^{2}$.  The variance-accounted-for in \verb+Y+ from \verb+YHAT+ is given by \verb+VAF_YHAT+, and defined as
\[ 1 - \frac{\sum_{i=1}^{n} (y_{i} - \hat{y}_{i})^{2}}{\sum_{i=1}^{n} (y_{i} - \bar{y})^{2}} \ , \]
\noindent where $\bar{y}$ is the mean value in $\mathbf{y}$.

\smallskip

The least-squares optimization strategy implemented in the four M-files of this section are all based on an iterative projection strategy developed by Dykstra (1983).  The first routine dealing with monotonic regression per se, \verb+monotonic_regression_dykstra.m+, is a direct competitor to one that would implement a ``pool adjacent violators'' algorithm; the latter has been the standard strategy ever since implemented by Kruskal (1964a,b) in his approach to nonmetric multidimensional scaling.  For example, the MATLAB Statistical Toolbox has an M-file for nonmetric multidimensional scaling, \verb+mdscale.m+. that relies on a (private) function, \verb+lsqisotonic.m+, which is a least-squares isotonic regression strategy built on the ``pool adjacent violators'' method.  To see how one of the four M-files of this section would work in \verb+mdscale.m+, the call to \verb+lsqisotonic.m+ could be replaced by one of these.  In effect, we can easily implement a nonmetric multidimensional scaling routine that would allow constraints beyond monotonic to convex, concave, or a combination of convex and concave.  Generally, these further constraints seem to ``smooth'' out the monotonic regression function quite dramatically; this has been the goal of using monotonic splines in exactly this same context (e.g., see Ramsay, 1988).  In effect, we have solved with these mechanisms rather directly and without splines, one of the major problems identified by Shepard (1974) in the Psychometric Society Presidential Address:


\bigskip


\emph{The problem that remains, however, is to impose the desired condition of smoothness in some general and nonarbitrary way without having to specify a particular functional form --- such as the polynomial (which can become nonmonotonic) or the exponential (which is often too  restrictive).} (p.\ 398)

\section{The Dykstra-Kaczmarz Method for Solving Linear (In)equality Constrained Least-Squares Tasks}

The Dykstra-Kaczmarz (DK) method for solving (in)equality constrained least-squares tasks, can be characterized as a very general iterative projection strategy that considers each linear constraint successively (and repeatedly), until convergence.  It serves as \emph{the} method for all of our least-squares fitting necessities in this and the previous three Toolboxes in the series.  We first review very cryptically, in the next section below, the basic argument for the iterative strategy; this then serves as the template that must be specialized for a particular data analysis context and followed in detail.  For example, when fitting structures such as ultrametrics, additive trees, anti-Robinson forms, and the like, the fitting procedure reviewed is directly programmed into an M-file.  A second section discusses a general MATLAB M-file based on converting a Fortran subroutine from Wollan and Dykstra (1987).  This is a general optimization routine that can be used in the spirit of other such M-files from the Optimization Toolbox.  It is written very generally, and not for just one particular application (as we have done, for example, in fitting ultrametrics with the dedicated M-file, \verb+ultrafit.m+).  It accepts general arguments to define the constrained least-squares task, such as the characterizing constraint matrix, and produces a solution.  At times, a general implementation such as this may be problematic if, say, the constraint matrix is too enormous in size.  In these cases, there may be no other choice than to program the specific application without storing the constraint matrix.

\subsection{A Review of the DK Strategy}

The Kaczmarz (1937) method deals only with equality constraints; Dykstra's (1983) method  generalizes the  restrictions that can be handled by Kaczmarz's strategy to the use of inequality constraints.

\smallskip


Kaczmarz's method can be characterized as follows:

\smallskip

Given $\mathbf{A} = \{a_{ij}\}$ of order $m \times n$, $\mathbf{x}' = \{x_{1},\ldots,x_{n}\}$, $\mathbf{b}' = \{b_{1},\ldots,b_{m}\}$, and assuming the linear system $\mathbf{Ax} = \mathbf{b}$ is consistent, define the set $C_{i} = \{\mathbf{x}  \ | \ \sum_{j=1}^{n} a_{ij} x_{j} = b_{i}, 1 \leq j \leq n\}$, for $1 \leq i \leq m$.  The projection of any $n \times 1$ vector $\mathbf{y}$ onto $C_{i}$ is simply $\mathbf{y} - (\mathbf{a}_{i}' \mathbf{y} - b_{i}) \mathbf{a}_{i}(\mathbf{a}_{i}' \mathbf{a}_{i})^{-1}$, where $\mathbf{a}_{i}' = \{a_{i1},\ldots,a_{in}\}$.  Beginning with a vector $\mathbf{x}_{0}$, and successively projecting $\mathbf{x}_{0}$ onto $C_{1}$, and that result onto $C_{2}$, and so on, and cyclically and repeatedly reconsidering projections onto the sets $C_{1},\ldots,C_{m}$, leads at convergence to a vector $\mathbf{x}_{0}^{*}$ that is closest to $\mathbf{x}_{0}$ (in vector 2-norm, so that $\sum_{j=1}^{n} (x_{0j} - x_{0j}^{*})^{2}$ is minimized) and $\mathbf{Ax}_{0}^{*} = \mathbf{b}$.  In short, Kaczmarz's method iteratively solves least-squares tasks subject to equality restrictions.

\bigskip

Dykstra's method can be characterized as follows:

\smallskip

Given $\mathbf{A} = \{a_{ij}\}$ of order $m \times n$, $\mathbf{x}_{0}' = \{x_{01},\ldots,x_{0n}\}$, $\mathbf{b}' = \{b_{1},\ldots,b_{m}\}$, and $\mathbf{w}' = \{w_{1},\ldots,w_{n}\}$, where $w_{j} > 0$ for all $j$, find $\mathbf{x}_{0}^{*}$ such that $\mathbf{a}_{i}' \mathbf{x}_{0}^{*} \leq b_{i}$ for $1 \leq i \leq m$ and $\sum_{j=1}^{n} w_{j} (x_{0j} - x_{0j}^{*})^{2} $ is minimized.  Again, (re)define the (closed convex) sets $C_{i} = \{ \mathbf{x} \ | \ \sum_{j=1}^{n} a_{ij} x_{j} \leq b_{i}, 1 \leq j \leq n\}$ and when a vector $\mathbf{y} \notin C_{i}$, its projection onto $C_{i}$ (in the metric defined by the weight vector $\mathbf{w}$) is $\mathbf{y} - (\mathbf{a}_{i}' \mathbf{y} - b_{i})\mathbf{a}_{i}\mathbf{W}^{-1}(\mathbf{a}_{i}'\mathbf{W}^{-1}\mathbf{a}_{i})^{-1}$, where $\mathbf{W}^{-1} = \mathrm{diag} \{w_{1}^{-1},\ldots,w_{n}^{-1}\}$. We again initialize the process with the vector $\mathbf{x}_{0}$ and each set $C_{1},\ldots,C_{m}$ is considered in turn.  If the vector being carried forward to this point when $C_{i}$ is (re)considered does not satisfy the constraint defining $C_{i}$, a projection onto $C_{i}$ occurs.  The sets $C_{1},\ldots,C_{m}$ are cyclically and repeatedly considered but with one difference from the operation of Kaczmarz's method --- each time a constraint set $C_{i}$ is revisited, any changes from the previous time $C_{i}$ was reached are first ``added back''.  This last process ensures convergence to a (globally) optimal solution $\mathbf{x}_{0}^{*}$.


\subsection{A General M-file for Solving Linear Inequality Constrained Least-Squares Tasks}

The Fortran subroutine that we have rewritten into an M-file from Wollan and Dykstra (1987) is called \verb+least_squares_dykstra.m+, and is meant to find a solution to the following problem:

\bigskip

\noindent suppose $\mathbf{g}$ is a given $n \times 1$ vector (the ``dependent'' vector);  $\mathbf{S}$  a given $n \times n$ positive-definite matrix (thus, it has an inverse; it may, for example, be a variance-covariance matrix, in which case we are minimizing Mahalanobis distances in the formulation below); $\mathbf{A}$ is a known $m \times n$ matrix where each row of $\mathbf{A}$ defines one of $m$ constraints; $\mathbf{b}$ is a known $m \times 1$ vector giving the right-hand-sides of the inequality constraints.  The task: find an $n \times 1$ vector $\mathbf{x}$, subject to $\mathbf{A}\mathbf{x} \le \mathbf{b}$, to minimize
\begin{equation}
(\mathbf{g} - \mathbf{x})'\mathbf{S}^{-1}(\mathbf{g} - \mathbf{x}) \ .
\end{equation}
\noindent Commonly, $\mathbf{S}$ will be the identity, in which case we have unweighted least-squares; if $\mathbf{S}$ is diagonal, weighted least-squares is obtained.  We also note that in the actual application of the M-file, equality constraints can be enforced directly.

\smallskip

The syntax for the M-file, \verb+least_squares_dykstra.m+, is as follows:
\begin{verbatim}
 [solution,kuhn_tucker,iterations,end_condition] = ...
    least_squares_dykstra(data,covariance,constraint_array, ...
    constraint_constant,equality_flag)
\end{verbatim}
\noindent Here, the input arguments are an $n \times 1$ vector, \verb+DATA+; an $n \times n$  positive-definite matrix, \verb+COVARIANCE+; an $m \times n$ constraint matrix,  \verb+CONSTRAINT_ARRAY+; the $m \times 1$ right-hand-side constraint vector,  \verb+CONSTRAINT_CONSTANT+; and an $m \times 1$ \verb+EQUALITY_FLAG+ vector with  values of 0 when a corresponding constraint is an inequality, and 1 if  an equality.  The weighted least-squares criterion (with weights defined by the  inverse of \verb+COVARIANCE+) is minimized by a \verb+SOLUTION+ that satisfies
 \verb+CONSTRAINT_ARRAY * SOLUTION+ being in \verb+EQUALITY_FLAG+ relation to
 \verb+CONSTRAINT_CONSTANT+.  As additional output arguments, there is a $m
 \times 1$ \verb+KUHN_TUCKER+ vector (useful for some applications); the number
 of \verb+ITERATIONS+ taken (maximum default value is \verb+ITERMAX+ $= 1.0e+04$, set in  the program), and an \verb+END_CONDITION+ flag: 0: no error; 1: \verb+ITERMAX+
 exceeded; 2: invalid constant; 3: invalid constraint function.

 \subsubsection{A Few Applications of least\_squares\_dykstra.m}

 There are a few very useful application of \verb+least_squares_dykstra.m+ that we review briefly here:

 \smallskip

 (A) Consider the minimization of
 \begin{equation}
(\mathbf{g} - \mathbf{x})'\mathbf{S}^{-1}(\mathbf{g} - \mathbf{x}) \ ,
\end{equation}
\noindent subject to
\[ \mathbf{x} = \sum_{i=1}^{m} \alpha_{i} \mathbf{d}_{i}, \ \alpha_{i} \ge 0 \ \mathrm{for} \ 1 \le i \le m, \ \]
\noindent and the specified collection of $n \times 1$ vectors, $\mathbf{d}_{1},\ldots,\mathbf{d}_{k}$.  To use the general M-file for this task, define  the constraint array to be,
\[ \mathbf{A} = \left[ \begin{array}{c}
                        \mathbf{d}_{1}'\mathbf{S}^{-1} \\

                           \vdots  \\

                            \mathbf{d}_{m}'\mathbf{S}^{-1} \\ \end{array} \right] \ .\]
\noindent Letting $\mathbf{x}^{*}$ denote the solution of the original task in (17), the solution here is given by $\mathbf{g} -  \mathbf{x}^{*}$, with the $\alpha_{i}$'s being 1/2 the values in the Kuhn-Tucker vector.  We might note that these fitting ideas were implemented in the M-file, \verb+partitionfit.m+, used heavily in the Cluster Analysis Toolbox to generalize the notion of an ultrametric.

\smallskip

(B) Consider the usual linear model formulation: $\mathbf{y} = \mathbf{X}\mathbf{\mbox{\boldmath{$\beta$}}} + \mathbf{\epsilon}$, where $\mathbf{y}$ is the $n \times 1$ vector of observations, $\mathbf{X}$ an $n \times m$ full-rank design matrix, and $\mathbf{\epsilon}$ contains $n$ independent, $N(0,\sigma^{2})$, random variables.  If we wish to impose linear inequality constraints on the least-squares estimate of the $m \times 1$ vector $\mathbf{\mbox{\boldmath{$\beta$}}}$, say, $\mathbf{\hat{\mbox{\boldmath{$\beta$}}}}$: $\mathbf{A}\mathbf{\hat{\mbox{\boldmath{$\beta$}}}} \le \mathbf{b}$, in our original problem formulation in (18), we can let $\mathbf{S}^{-1} = \mathbf{X}'\mathbf{X}$, $\mathbf{g} = (\mathbf{X}'\mathbf{X})^{-1}\mathbf{X}'\mathbf{y}$, and use the given constraint array and vector, $\mathbf{A}$ and $\mathbf{b}$.

\smallskip

If we wish only nonnegativity of $\mathbf{\hat{\mbox{\boldmath{$\beta$}}}}$, then (18) can be used with $\mathbf{g} = \mathbf{y}$, $\mathbf{S} = \mathbf{I}$, and the $\mathbf{d}_{1}, \ldots, \mathbf{d}_{m}$ given by the $k$ columns of $\mathbf{X}$.  When halved, the minimizing $\alpha_{i}$'s provide the desired nonnegative estimate, $\mathbf{\hat{\mbox{\boldmath{$\beta$}}}} \ge \mathbf{0}$.

\section{Matrix Color Coding and Presentation}

 A current method of data presentation is through the use of color, where a collection of varying numerical values is mapped into a corresponding collection of colors that can then be displayed.  A \verb+colormap+ is an $m \times 3$ matrix of real numbers between 0.0 and 1.0, with each row giving an RGB (Red/Green/Blue) vector defining a single color.  For example, if \verb+map+ is the name of the $m \times 3$ matrix, then the $k^{th}$ row, \verb+map(k,:) = [r(k) g(k) b(k)]+, provides the intensities of red, green, and blue components for the $k^{th}$ color.

\smallskip

A number of built-in colormaps are available in MATLAB (or, alternatively, custom colormaps can be constructed by a user, possibly through the MATLAB GUI, \verb+colormapeditor+); all of these ready-made colormaps accept a value of $m$ for the number of rows (number of colors) of the colormap.  For example, the command, \verb+colormap(bone(256))+, produces a (built-in) \verb+bone+ colormap with 256 colors; the latter is a grayscale with a higher value of the blue component, adding an ``electronic'' look to the grayscale images.  Asking for MATLAB help on ``colormap'' will produce the collection of supported colormaps (e.g., \verb+bone+, \verb+gray+, \verb+copper+, \verb+spring+, among many others).

\smallskip

The MATLAB function, \verb+pcolor+, is used to produce our checkerboard colorcoded plot of the entries in a given matrix.  The minimum and maximum matrix entries are assigned the first and last colors in the colormap, with the remaining entries determined by a linear mapping from the numerical values to the colormap elements.  The MATLAB command, \verb+colorbar+, reproduces a legend giving a visual correspondence between color and numerical value.  Also, the MATLAB command, \verb+axis ij+, puts the representation into ``matrix axes mode'', where the coordinate system origin is at the upper-left corner.  As is standard for a matrix, the $i$ axis is vertical and numbered from top to bottom; the $j$ axis is horizontal and numbered from left to right.

\smallskip

The M-file we have written, \verb+matrix_colorcode.m+, has syntax:
\begin{verbatim}
matrix_colorcode(datamatrix,rowperm,colperm,cmap)
\end{verbatim}
\noindent Here,  a color representation  is constructed for the values in an
$n \times m$ matrix, \verb+DATAMATRIX+.  The rows and columns of \verb+DATAMATRIX+ are
permuted by \verb+ROWPERM+ and \verb+COLPERM+, respectively, with  \verb+CMAP+ the input
colormap for the representation (e.g., \verb+bone(256)+).

\smallskip

There are several other uses of the \verb+pcolor+ command that are interesting from a data analytic perspective.  One is to produce a movie that traces the course of some optimization method.  For example, the various quadratic assignment routines of the earlier Unidimensional Scaling Toolbox (e.g., \verb+order.m+, \verb+pairwiseqa.m+, \verb+rotateqa.m+, \verb+insertqa.m+), all produce a cell array of permutations that transform a symmetric data matrix to be as close as possible to a given target matrix.  To show the effects of these permutations in a colorcoded manner, we have the M-file, \verb+matrix_movie.m+, with the syntax:
\begin{verbatim}
matrix_movie(datamatrix,perms,numperms,cmap)
\end{verbatim}
\noindent A color movie is constructed of the effects of a series
of permutations on an $n \times n$ symmetric   proximity matrix,
\verb+DATAMATRIX+; \verb+PERMS+ is
a cell array containing \verb+NUMPERMS+ permutations; \verb+CMAP+
is the input colormap used for the representation.  The actual movie is produced and played based on the MATLAB commands of \verb+getframe+ (for capturing the movie frames) and \verb+movie+ (for replaying the recorded frames).

\section{The Representation of Proximity Matrices by Structures Dependent on Order (Only)}

The third part of the recent monograph (2006) by Hubert, Arabie, and Meulman (\emph{The Structural Representation of Proximity Matrices with MATLAB}, SIAM), included three chapters devoted to order-theoretic representational structures for proximity matrices.  These chapters are given below in a slightly edited form to eliminate dependence on the actual monograph.  Some of the terms that will be introduced will be familiar from the earlier Toolboxes in this series (e.g., an anti-Robinson (AR) form), but others are unique here (in fact, this is the major reason for including this material in the Miscellany Toolbox): strongly anti-Robinson (SAR); circular anti-Robinson (CAR); circular strongly anti-Robinson (CSAR).  Also, multistructural analyses are discussed as well as extensions to two-mode proximity data.

\smallskip

This section concentrates on an alternative approach to understanding what a given proximity matrix may be depicting about the objects on which it was constructed, and one that does not require a prior commitment to the sole use of either some form of dimensional model (as in nonmetric multidimensional scaling (NMDS)), or one that is strictly classificatory (as in the use of a partition hierarchy and the implicit fitting of an ultrametric that serves as the representational mechanism for the hierarchical clustering).  The method of  analysis is based on approximating a given proximity matrix additively by a sum of matrices, where each component in the sum is subject to specific patterning restrictions on its entries.  The restrictions imposed on each component of the decomposition (to be referred to as matrices with AR, SAR, CAR, or CSAR forms) are very general and encompass interpretations that might be dimensional, or classificatory, or  some combination of both (e.g., through object classes that are themselves placed dimensionally in some space). Thus, as one special case --- and particularly when an (optimal) transformation of the proximities is also permitted (as we will generally allow), proximity matrices that are well interpretable through NMDS should also be interpretable through an additive decomposition of the (transformed) proximity matrix.  Alternatively, when classificatory structures of various kinds might underlie a set of proximities (and the direct use of NMDS  could possibly lead to a degeneracy), additive decompositions may still provide an analysis strategy for elucidating the structure.

\subsection{Anti-Robinson (AR) Matrices for Symmetric Proximity Data}



Denoting an arbitrary symmetric $n \times n$ matrix by $\mathbf{A} = \{a_{ij}\}$, where the main diagonal entries are considered irrelevant and assumed to be zero (i.e., $a_{ii} = 0$ for $1 \leq i \leq n$), $\mathbf{A}$ is said to have an anti-Robinson (AR) form if after some reordering of the rows and columns of $\mathbf{A}$, the entries within each row and column have a distinctive pattern: moving away from the zero main diagonal entry within any row or any column, the entries never decrease.  Generally, matrices having AR forms can appear both in spatial representations for a set of proximities as functions of the absolute differences in coordinate values along some axis, or for classificatory structures that are characterized through an ultrametric.

\smallskip

To illustrate, we first let $\mathbf{P} = \{p_{ij}\}$ be a given $n \times n$ proximity (dissimilarity) matrix among the distinct pairs of $n$ objects in a set $S =  \{O_{1},O_{2},\ldots,O_{n}\}$ (where $p_{ii} = 0$ for $1 \leq i \leq n$).  Then, suppose, for example, a two-dimensional Euclidean representation is possible for $\mathbf{P}$ and its entries are very well representable by the distances in this space, so \[ p_{ij} \approx \sqrt{(x_{1i}    - x_{1j})^{2} + (x_{2i}  - x_{2j})^{2}} \  , \] where $x_{ki}$ and $x_{kj}$ are the coordinates on the $k^{th}$ axis (for $k = 1$ and $2$) for objects $O_{i}$ and $O_{j}$ (and the symbol $\approx$  is used to indicate approximation).  Here, a simple monotonic transformation (squaring) of the proximities should then be fitted well by the sum of two matrices both having AR forms, i.e., \[ \{p_{ij}^{2}\} \approx \{(x_{1i} - x_{1j})^{2}\} + \{(x_{2i} - x_{2j})^{2}\} . \]  In a classificatory framework, if $\{p_{ij}\}$ were well representable, say, as a sum of two matrices, $\mathbf{A}_{1} = \{a_{ij}^{(1)}\}$ and $\mathbf{A}_{2} = \{a_{ij}^{(2)}\}$, each satisfying the ultrametric inequality, i.e., $a_{ij}^{(k)} \leq \max \{a_{ih}^{(k)}, a_{hj}^{(k)}\}$ for $k = 1$ and $2$, then \[\{p_{ij}\} \approx \{a_{ij}^{(1)}\} + \{a_{ij}^{(2)}\} ,\] and each of the constituent matrices can be reordered to display an AR form.  As shown in the Cluster Analysis Toolbox, any matrix whose entries satisfy the ultrametric inequality can be represented by a sequence of partitions that are hierarchically related.

\smallskip

Given some proximity matrix $\mathbf{P}$, the task of approximating it as a sum of matrices each having an AR form is implemented through an iterative optimization strategy based on a least-squares loss criterion that is discussed in detail by Hubert and Arabie (1994). Given the manner in which the optimization process is carried out sequentially, \emph{each successive AR matrix} in any decomposition generally accounts for less and less of the patterning of the original proximity information (and very analogous to what is typically observed in a principal component decomposition of a covariance matrix).  In fact, it has been found empirically that for the many data sets we have analyzed, only a very small number of such AR matrices are ever necessary to represent almost all of the patterning in the given proximities.  As a succinct summary that we could give to this empirical experience: no more than three AR matrices are ever necessary;  two are usually sufficient; and sometimes one will suffice.

\smallskip


The substantive challenge that remains, once a well-fitting decomposition is found for a given proximity matrix, is to interpret substantively what each term in the decomposition might be depicting.  The strategy  that could be followed would approximate each separate AR matrix by ones having a more restrictive form, and  usually those representing some type of unidimensional scale  (a dimensional interpretation) or partition hierarchy (a classificatory interpretation).

 \subsubsection{Incorporating Transformations}

One generalization that we will now allow to what has already been discussed in the literature for fitting sums of AR matrices to a proximity matrix $\mathbf{P}$, is the possible inclusion of an (optimal) transformation of the proximities.  Thus, instead of just representing $\mathbf{P}$ as a sum of $K$ matrices (and generally, for $K$ very small) that we might denote as $\mathbf{A}_{1} + \cdots + \mathbf{A}_{K}$, where each $\mathbf{A}_{k}$, $1 \leq k \leq K$, has an AR form, an (optimally) transformed matrix $\tilde{\mathbf{P}} = \{\tilde{p}_{ij}\}$ will be fitted by such a sum, say, $\tilde{\mathbf{A}}_{1} + \cdots + \tilde{\mathbf{A}}_{K}$, where the entries in $\tilde{\mathbf{P}}$ are monotonic with respect to those in $\mathbf{P}$, i.e., for all $O_{i}, O_{j}, O_{k}, O_{l} \in S$, $p_{ij} < p_{kl} \Rightarrow \tilde{p}_{ij} \leq \tilde{p}_{kl}$.  In the sequel we will rely on the M-file, \verb+proxmon.m+, which constructs optimal monotonic transformations by the same method of isotonic regression commonly used in NMDS (although using a different type of algorithm based on the Dykstra-Kaczmarz iterative projection strategy).  The method is Kruskal's (1964a,b) primary approach to tied proximities in $\mathbf{P}$ that are allowed to be untied after transformation.  Such transformations, for example, form the default option in the implementation of NMDS in the program KYST-2A (Kruskal, Young, and Seery, 1977) and in  SYSTAT (Wilkinson, 1988).

\smallskip


The process of finding $\tilde{\mathbf{P}}$ and $\tilde{\mathbf{A}}_{1} + \cdots + \tilde{\mathbf{A}}_{K}$ proceeds iteratively, with the original proximity matrix $\mathbf{P}$ first fit by $\mathbf{A}_{1} + \cdots + \mathbf{A}_{K}$; a subsequent optimal (monotonic) transformation of $\mathbf{P}$ (through a least-squares approximation to $\mathbf{A}_{1} + \cdots + \mathbf{A}_{K}$) is identified, which is then refitted by the matrix sum.  In many cases, this whole process can now be cycled through iteratively until convergence, i.e., a sequential fitting and refitting of the optimally transformed proximities and its representation as a sum of matrices each having an AR form.

\smallskip

 In some contexts, however (particularly when fitting a single AR matrix [i.e., when $K = 1$]), it is probably best not to proceed to a complete convergence but instead to terminate the process after only a single optimal monotonic transformation of $\mathbf{P}$ is identified and then to refit by a matrix sum.  This usage will be referred to as a single iteration optimal transformation (SIOT).  If carried through to convergence, a perfect representation may be obtained but only at the expense of losing almost all  the patterning contained within the original proximity matrix.  For example, in fitting a single AR matrix, the optimal transformation identified after convergence might consist of just two values, with one corresponding to the smallest proximity in the original matrix and all others equal.  Although technically permissible since this situation does reflect a perfect AR form, most of the detail present in the original proximity matrix is also lost.  Difficulties with such so-called degeneracies have been pointed out by Carroll (1992),  particularly when faced with fitting classificatory structures to a given proximity matrix.


\subsubsection{Interpreting the Structure of an AR matrix}

In representing a proximity matrix $\mathbf{P}$ as a sum, $\mathbf{A}_{1} + \cdots + \mathbf{A}_{K}$ (or an optimal transformation $\tilde{\mathbf{P}}$ as $\tilde{\mathbf{A}}_{1} + \cdots + \tilde{\mathbf{A}}_{K}$), the interpretive task remains to explain substantively what each term of the decomposition might be depicting.  We suggest four possible strategies below, with the first two attempting to understand the structure of an AR matrix directly and without much loss of detail; the last two require the imposition of strictly parameterized approximations in the form of either an ultrametric or a unidimensional scale.  In the discussion below, $\mathbf{A} = \{a_{ij}\}$ will be assumed to have an AR form that is displayed by the given row and column order.

\bigskip

(A) Complete representation and reconstruction through a collection of subsets and associated subset diameters:

\smallskip

The entries in any AR matrix $\mathbf{A}$ can be reconstructed exactly through a collection of $M$ subsets of the original object set $S = \{O_{1},\ldots,O_{n}\}$, denoted by $S_{1},\ldots,S_{M}$,  and where $M$ is determined by the particular pattern of tied entries, if any,  in $\mathbf{A}$.  These $M$ subsets have the following characteristics:

(i) each $S_{m}$, $1 \leq m \leq M$, consists of a sequence of (two or more) consecutive integers so that $M \leq n(n-1)/2$.  (This bound holds because the number of different subsets having consecutive integers for any given fixed ordering is $n(n-1)/2$, and will be achieved if all the entries in the AR matrix $\mathbf{A}$ are distinct).

(ii) each $S_{m}$, $1 \leq m \leq M$, has a diameter, denoted by $d(S_{m})$, so that for all object pairs within $S_{m}$, the corresponding entries in $\mathbf{A}$ are less than or equal to the diameter.  The subsets, $S_{1},\ldots,S_{M}$, can be assumed ordered as $d(S_{1}) \leq d(S_{2}) \leq \cdots \leq d(S_{M})$, and if $S_{m} \subseteq S_{m'}$, $d(S_{m}) \leq d(S_{m'})$.


(iii) each entry in $\mathbf{A}$ can be reconstructed from $d(S_{1}),\ldots,d(S_{M})$, i.e., for $1 \leq i,j \leq n$, \[ a_{ij} = \min_{1 \leq m \leq M} \{d(S_{m}) \mid O_{i}, O_{j} \in S_{m} \} ,\]
\noindent so that the minimum diameter for subsets containing an object pair $O_{i}, O_{j} \in S$ is equal to $a_{ij}$.    Given $\mathbf{A}$, the collection of subsets $S_{1},\ldots,S_{M}$ and their diameters can be identified by inspection through the use of an increasing threshold that starts from the smallest entry in $\mathbf{A}$, and observing which subsets containing contiguous objects emerge from this process.  The substantive interpretation of what $\mathbf{A}$ is depicting reduces to explaining why those subsets with the smallest diameters are so homogenous.  For convenience of reference, the subsets $S_{1},\ldots,S_{M}$ will be referred to as the set of \emph{AR reconstructive subsets}.

\bigskip

(B) Representation by a strongly anti-Robinson matrix:

\smallskip
 \index{B}{anti-Robinson forms}
If the matrix $\mathbf{A}$ has a somewhat more restrictive form than just being AR, and is also   \emph{strongly} anti-Robinson (SAR), a convenient graphical representation can be given to the collection of AR reconstructive subsets $S_{1},\ldots,S_{M}$ and their diameters, and how they can serve to retrieve $\mathbf{A}$.  Specifically, $\mathbf{A}$ is said to be strongly anti-Robinson (SAR) if (considering the above-diagonal entries of $\mathbf{A}$) whenever two entries in adjacent columns are equal ($a_{ij} = a_{i(j+1)}$), those in the same two adjacent columns in the previous row are also equal ($a_{(i-1)j} = a_{(i-1)(j+1)}$ for $1 \leq i-1 < j \leq n-1$); also, whenever two entries in adjacent rows are equal ($a_{ij} = a_{(i+1)j}$), those in the same two adjacent rows in the succeeding column are also equal ($a_{i(j+1)} = a_{(i+1)(j+1)}$ for $2 \leq i+1 < j \leq n-1$).

\smallskip

 When $\mathbf{A}$ is SAR, the collection of subsets, $S_{1},\ldots,S_{M}$, and their diameters, and how these serve to reconstruct $\mathbf{A}$ can be modeled graphically.  The internal nodes (represented by  solid circles) in each of these figures are at a height equal to the diameter of the respective subset; the consecutive objects forming that subset are identifiable by downward paths from the internal nodes to the terminal nodes corresponding to the objects in $S = \{O_{1},\ldots,O_{n}\}$ (represented by labeled open circles).  An entry $a_{ij}$ in $\mathbf{A}$ can be reconstructed as the minimum node height of a subset for which a path can be constructed from $O_{i}$ up to that internal node and then back down to $O_{j}$. (To prevent undue graphical ``clutter'', only the most homogenous subsets from $S_{1},\ldots,S_{M}$ having the smallest diameters should actually be included in the graphical representation of an SAR matrix; each figure would explicitly show only how the smallest entries in $\mathbf{A}$ can be reconstructed, although each could be easily extended to include all of $\mathbf{A}$.  The calibrated vertical axis in such figures could routinely include the heights at which the additional internal nodes would have to be placed to effect such a complete reconstruction.)

\smallskip

Given an arbitrary AR matrix $\mathbf{A}$, a least-squares SAR approximating matrix to $\mathbf{A}$ can be found using the heuristic optimization search strategy  developed in Hubert, Arabie, and Meulman (1998), and illustrated in a section to follow.  This latter source also discusses in detail (through counterexample) why strongly AR conditions need to be imposed to obtain a consistent graphical representation.

\bigskip

(C) Representation by a unidimensional scale:

\smallskip

To obtain greater graphical simplicity for an eventual substantive interpretation than offered by an SAR matrix, one possibility is to use approximating unidimensional scales.  To be explicit, one very simple form that an AR matrix $\mathbf{A}$ may assume is interpretable by a single dimension and through a unidimensional scale in which the entries have the parameterized form,
 $\mathbf{A} = \{a_{ij}\} = \{ \mid x_{j} - x_{i} \mid + \ c \}$, where the coordinates are ordered as $x_{1} \leq x_{2} \leq \cdots \leq x_{n}$ and $c$ is an estimated constant.  Given any proximity matrix, a least-squares approximating unidimensional scale can be obtained through the optimization strategies of the Unidimensional Scaling Toolbox,  and would be one (dimensional) method that could be followed in attempting to interpret what a particular AR component of a decomposition might be revealing.

\bigskip

(D)  Representation by an ultrametric:

\smallskip


A second simple form that an AR matrix $\mathbf{A}$ could have is strictly classificatory in which the entries in $\mathbf{A}$ satisfy the ultrametric condition: $a_{ij} \leq \max \{a_{ik}, a_{jk}\}$ for all $O_{i}, O_{j}, O_{k} \in S$.  As a threshold is increased from the smallest entry in $\mathbf{A}$, a sequence of partitions of $S$ is identified in which each partition is constructed from the previous one by uniting pairs of subsets from the latter.  A partition identified at a given threshold level has equal values in $\mathbf{A}$ between each given pair of subsets, and all the within subset values are not greater than the between subset values.  The reconstructive subsets $S_{1},\ldots,S_{M}$ that would represent the AR matrix $\mathbf{A}$ are now the new subsets that are formed in the sequence of partitions, and have the property that if $d(S_{m}) \leq d(S_{m'})$, then $S_{m} \subseteq S_{m'}$ or $S_{m} \cap S_{m'} = \oslash$.
Given any proximity matrix, a least-squares approximating ultrametric can be constructed by the heuristic optimization routines developed in the Cluster Analysis Toolbox, and would be another (classificatory) strategy for interpreting what a particular AR component of a decomposition might be depicting.  As might be noted, there are generally $n-1$ subsets (each of size greater than one) in the collection of reconstructive subsets for any ultrametric, and thus $n-1$  values need to be estimated in finding the least-squares approximation (which is the same number   needed for a least-squares approximating unidimensional scale, based on obtaining the $n-1$ non-negative separation values between $x_{i}$ and $x_{i+1}$ for $1 \leq i \leq n-1$).




\subsection{Fitting a Given AR Matrix in the $L_{2}$-Norm}

The function M-file, \verb+arobfit.m+, fits an anti-Robinson matrix using iterative projection to a symmetric proximity matrix in the $L_{2}$-norm.  The usage syntax is of the form
\begin{verbatim}
[fit,vaf] = arobfit(prox,inperm)
\end{verbatim}
\noindent where \verb+PROX+ is the input proximity matrix ($n \times n$ with a zero main diagonal
 and a dissimilarity interpretation); \verb+INPERM+ is a given permutation of the first $n$ integers; \verb+FIT+ is the least-squares optimal matrix (with variance-accounted-for
 of \verb+VAF+) to \verb+PROX+ having an anti-Robinson form for the row and column
 object ordering given by \verb+INPERM+.  A recording of a MATLAB  session using the \verb+number.dat+ data file and object ordering given by the identity permutation follows:

 \tiny

\begin{verbatim}
load number.dat
inperm = 1:10

inperm =

     1     2     3     4     5     6     7     8     9    10

[fit,vaf] = arobfit(number,inperm)

fit =

  Columns 1 through 6

         0    0.4210    0.5840    0.6965    0.6965    0.7960
    0.4210         0    0.2840    0.3460    0.6170    0.6170
    0.5840    0.2840         0    0.2753    0.2753    0.5460
    0.6965    0.3460    0.2753         0    0.2753    0.3844
    0.6965    0.6170    0.2753    0.2753         0    0.3844
    0.7960    0.6170    0.5460    0.3844    0.3844         0
    0.7960    0.6940    0.5460    0.3844    0.3844    0.3844
    0.8600    0.6940    0.5853    0.5853    0.5530    0.4000
    0.8600    0.7413    0.5853    0.5853    0.5530    0.5530
    0.8600    0.7413    0.7413    0.5853    0.5853    0.5853

  Columns 7 through 10

    0.7960    0.8600    0.8600    0.8600
    0.6940    0.6940    0.7413    0.7413
    0.5460    0.5853    0.5853    0.7413
    0.3844    0.5853    0.5853    0.5853
    0.3844    0.5530    0.5530    0.5853
    0.3844    0.4000    0.5530    0.5853
         0    0.3857    0.3857    0.3857
    0.3857         0    0.3857    0.3857
    0.3857    0.3857         0    0.3857
    0.3857    0.3857    0.3857         0


vaf =

    0.6979
\end{verbatim}

\large

\subsubsection{Fitting the (In)-equality Constraints Implied by a Given Matrix in the $L_{2}$-Norm}


At times it may be useful to fit  through iterative projection a given set of equality and
 inequality constraints (as represented by the equalities and
 inequalities present among the entries in a given target matrix) to a symmetric proximity matrix in the $L_{2}$-norm.  Whenever the target matrix is AR in form already, the resulting fitted matrix would also be AR in form; more generally, however, the M-function, \verb+targfit.m+, could be used with any chosen target matrix.  The usage follows the form
\begin{verbatim}
[fit,vaf] = targfit(prox,targ)
\end{verbatim}
 \noindent where, as usual, \verb+PROX+ is the input proximity matrix (with a zero main diagonal
 and a dissimilarity interpretation); \verb+TARG+ is a matrix of the same size as \verb+PROX+;
 \verb+FIT+ is the least-squares optimal matrix (with variance-accounted-for
 of \verb+VAF+) to \verb+PROX+ satisfying the equality and
 inequality constraints implicit among all the entries in \verb+TARG+.
An example follows in which the given target matrix is a distance matrix (having an AR form) between equally-spaced object placements along a line; the resulting fitted matrix obviously has an AR form as well:

\tiny

\begin{verbatim}
load number.dat
[fit,vaf] = targfit(number,targlin(10))

fit =

  Columns 1 through 6

         0    0.3714    0.3714    0.5363    0.5363    0.6548
    0.3714         0    0.3714    0.3714    0.5363    0.5363
    0.3714    0.3714         0    0.3714    0.3714    0.5363
    0.5363    0.3714    0.3714         0    0.3714    0.3714
    0.5363    0.5363    0.3714    0.3714         0    0.3714
    0.6548    0.5363    0.5363    0.3714    0.3714         0
    0.6548    0.6548    0.5363    0.5363    0.3714    0.3714
    0.7908    0.6548    0.6548    0.5363    0.5363    0.3714
    0.7908    0.7908    0.6548    0.6548    0.5363    0.5363
    0.8500    0.7908    0.7908    0.6548    0.6548    0.5363

  Columns 7 through 10

    0.6548    0.7908    0.7908    0.8500
    0.6548    0.6548    0.7908    0.7908
    0.5363    0.6548    0.6548    0.7908
    0.5363    0.5363    0.6548    0.6548
    0.3714    0.5363    0.5363    0.6548
    0.3714    0.3714    0.5363    0.5363
         0    0.3714    0.3714    0.5363
    0.3714         0    0.3714    0.3714
    0.3714    0.3714         0    0.3714
    0.5363    0.3714    0.3714         0


vaf =

    0.5105
\end{verbatim}

\large

\subsection{Finding an AR Matrix in the $L_{2}$-Norm}

The \emph{fitting} of a given AR matrix by the M-function, \verb+arobfit.m+, requires the presence of an initial permutation to direct the optimization process.  Thus, the \emph{finding} of a best-fitting AR matrix reduces to the identification of an appropriate object permutation to use \emph{ab initio}.  We suggest the adoption of \verb+order.m+, which carries out an iterative Quadratic Assignment (QA) maximization task using
 a given square, $n \times n$, proximity matrix \verb+PROX+ (with a zero main diagonal and
 a dissimilarity interpretation).  Three separate local operations are used to permute
 the rows and columns of the proximity matrix to maximize the cross-product
 index with respect to a given square target matrix \verb+TARG+: (a)
 pairwise interchanges of objects in the permutation defining the row and column
 order of the square proximity matrix; (b) the insertion of from 1 to \verb+KBLOCK+
 (which is less than or equal to $n-1$) consecutive objects in
 the permutation defining the row and column order of the data matrix; and (c) the
 rotation of from 2 to \verb+KBLOCK+ (which is less than or equal to $n-1$) consecutive objects in
 the permutation defining the row and column order of the data matrix.  The usage syntax has the form
\begin{verbatim}
[outperm,rawindex,allperms,index] =  ...
    order(prox,targ,inperm,kblock)
\end{verbatim}
\noindent where \verb+INPERM+ is the input beginning permutation (a permutation of the first $n$ integers);
 \verb+OUTPERM+ is the final permutation of \verb+PROX+ with the cross-product index \verb+RAWINDEX+
 with respect to \verb+TARG+. The cell array \verb+ALLPERMS+  contains \verb+INDEX+
 entries corresponding to all the
 permutations identified in the optimization from \verb+ALLPERMS{1}+ = \verb+INPERM+ to
 \verb+ALLPERMS{INDEX}+ = \verb+OUTPERM+.

 \smallskip

A recording of a MATLAB  session using \verb+order.m+ is listed below with the beginning \verb+INPERM+ given as the identity permutation, \verb+TARG+ by an equally-spaced object placement along a line, and \verb+KBLOCK+ = 3.  Using the generated \verb+OUTPERM+, \verb+arobfit.m+ is then invoked to fit an AR form having final \verb+VAF+ of .7782.


\tiny

\begin{verbatim}
load number.dat
targlinear = targlin(10);
[outperm,rawindex,allperms,index] = ...
    order(number,targlinear,1:10,3);

outperm

outperm =

     1     2     3     5     4     6     7     9    10     8

rawindex

rawindex =

  206.4920

index

index =

     4

[fit, vaf] = arobfit(number, outperm)

fit =

  Columns 1 through 6

         0    0.4210    0.5840    0.6840    0.7090    0.7960
    0.4210         0    0.2840    0.4960    0.4960    0.5880
    0.5840    0.2840         0    0.0590    0.3835    0.4928
    0.6840    0.4960    0.0590         0    0.3835    0.3985
    0.7090    0.4960    0.3835    0.3835         0    0.3750
    0.7960    0.5880    0.4928    0.3985    0.3750         0
    0.7960    0.7357    0.4928    0.3985    0.3750    0.3750
    0.8210    0.7357    0.4928    0.4928    0.4928    0.4928
    0.8500    0.7357    0.7357    0.6830    0.4928    0.4928
    0.9090    0.7357    0.7357    0.7357    0.5920    0.4928

  Columns 7 through 10

    0.7960    0.8210    0.8500    0.9090
    0.7357    0.7357    0.7357    0.7357
    0.4928    0.4928    0.7357    0.7357
    0.3985    0.4928    0.6830    0.7357
    0.3750    0.4928    0.4928    0.5920
    0.3750    0.4928    0.4928    0.4928
         0    0.3460    0.3460    0.4253
    0.3460         0    0.3460    0.4253
    0.3460    0.3460         0    0.4253
    0.4253    0.4253    0.4253         0


vaf =

    0.7782
\end{verbatim}

\large

The M-file, \verb+arobfnd.m+ is our preferred method for actually identifying a single AR form, and incorporates an initial equally-spaced target and uses the iterative QA routine of \verb+order.m+ to generate better permutations; the obtained AR forms are then used as new targets against which possibly even better permutations might be identified, until convergence (i.e., the identified permutations remain the same).  The syntax is as follows:
\begin{verbatim}
[find, vaf, outperm] = arobfnd(prox, inperm, kblock)
\end{verbatim}
\noindent where \verb+PROX+ is the input proximity matrix ($n \times n$ with a zero main diagonal
 and a dissimilarity interpretation);
 \verb+INPERM+ is a given starting permutation of the first $n$ integers;
 \verb+FIND+ is the least-squares optimal matrix (with variance-accounted-for
 of \verb+VAF+) to \verb+PROX+ having an anti-Robinson form for the row and column
 object ordering given by the ending permutation \verb+OUTPERM+; \verb+KBLOCK+
 defines the block size in the use the iterative quadratic assignment
 routine.

\smallskip


  As seen from the example below, and starting from a random initial permutation, the same AR form is found as with just one application of \verb+order.m+ reported above.

  \tiny

\begin{verbatim}
[find, vaf, outperm] = arobfnd(number, randperm(10), 1);

vaf =

    0.7782


outperm =

     8    10     9     7     6     4     5     3     2     1
\end{verbatim}

\large

\subsection{Fitting and Finding a Strongly Anti-Robinson (SAR) Matrix in the $L_{2}$-Norm}

The M-functions, \verb+sarobfit.m+ and \verb+sarobfnd.m+, are direct analogues of \verb+arobfit.m+ and \verb+arobfnd.m+, respectively, but are concerned with fitting and finding \emph{strongly} anti-Robinson forms.  The syntax for \verb+sarobfit.m+, which fits a strongly anti-Robinson matrix using iterative projection to
 a symmetric proximity matrix in the $L_{2}$-norm, is
\begin{verbatim}
[fit, vaf] = sarobfit(prox, inperm)
\end{verbatim}
\noindent where, again, \verb+PROX+ is the input proximity matrix ($n \times n$ with a zero main diagonal
 and a dissimilarity interpretation); \verb+INPERM+ is a given permutation of the first $n$ integers;
 \verb+FIT+ is the least-squares optimal matrix (with variance-accounted-for
 of \verb+VAF+) to \verb+PROX+ having a strongly anti-Robinson form for the row and column
 object ordering given by \verb+INPERM+.

 \smallskip

An example follows using the same identity permutation as was implemented in fitting an AR form with \verb+arobfit.m+; as might be expected from using the more restrictive strongly anti-Robinson form, the \verb+VAF+ drops to .6128 from .6979.

\tiny

\begin{verbatim}
load number.dat
[fit,vaf] = sarobfit(number,1:10)

fit =

  Columns 1 through 6

         0    0.4210    0.5840    0.6965    0.6965    0.7960
    0.4210         0    0.2840    0.4960    0.4960    0.6730
    0.5840    0.2840         0    0.2753    0.2753    0.4553
    0.6965    0.4960    0.2753         0    0.2753    0.4553
    0.6965    0.4960    0.2753    0.2753         0    0.3977
    0.7960    0.6730    0.4553    0.4553    0.3977         0
    0.7960    0.6730    0.4553    0.4553    0.3977    0.3977
    0.8600    0.6820    0.6050    0.6050    0.5557    0.5557
    0.8600    0.6820    0.6050    0.6050    0.5557    0.5557
    0.8600    0.6820    0.6050    0.6050    0.5557    0.5557

  Columns 7 through 10

    0.7960    0.8600    0.8600    0.8600
    0.6730    0.6820    0.6820    0.6820
    0.4553    0.6050    0.6050    0.6050
    0.4533    0.6050    0.6050    0.6050
    0.3977    0.5557    0.5557    0.5557
    0.3977    0.5557    0.5557    0.5557
         0    0.3857    0.3857    0.3857
    0.3857         0    0.3857    0.3857
    0.3857    0.3857         0    0.3857
    0.3857    0.3857    0.3857         0


vaf =

    0.6128
\end{verbatim}

\large

The M-function, \verb+sarobfnd.m+,  finds and fits a strongly anti-Robinson matrix using iterative projection to
 a symmetric proximity matrix in the $L_{2}$-norm based on a permutation
 identified through the use of iterative quadratic assignment.  The function has the expected syntax
\begin{verbatim}
[find, vaf, outperm] = sarobfnd(prox, inperm, kblock)
\end{verbatim}
 \noindent where, again, \verb+PROX+ is the input proximity matrix ($n \times n$ with a zero main diagonal
 and a dissimilarity interpretation); \verb+INPERM+ is a given starting permutation of the first $n$ integers;
 \verb+FIND+ is the least-squares optimal matrix (with variance-accounted-for
 of \verb+VAF+) to \verb+PROX+ having a strongly anti-Robinson form for the row and column
 object ordering given by the ending permutation \verb+OUTPERM+. As usual, \verb+KBLOCK+
 defines the block size in the use the iterative quadratic assignment
 routine.

 \smallskip

In the MATLAB  recording below, and starting from a random permutation, a strongly anti-Robinson form is found with a \verb+VAF+ of .7210 (an expected drop from the value of .7782 for the anti-Robinson form found using \verb+arobfnd.m+).


\tiny


\begin{verbatim}
[find,vaf,outperm] = sarobfnd(number,randperm(10),1)

find =

  Columns 1 through 7

         0    0.4210    0.5840    0.6965    0.6965    0.7960
    0.4210         0    0.2840    0.4960    0.4960    0.6730
    0.5840    0.2840         0    0.0590    0.3835    0.4723
    0.6965    0.4960    0.0590         0    0.3835    0.4723
    0.6965    0.4960    0.3835    0.3835         0    0.3750
    0.7960    0.6730    0.4723    0.4723    0.3750         0
    0.7960    0.6730    0.4723    0.4723    0.3750    0.3750
    0.8355    0.7080    0.5714    0.5714    0.4275    0.4275
    0.8355    0.7080    0.5714    0.5714    0.5714    0.5714
    0.9090    0.7227    0.7227    0.7227    0.5714    0.5714

  Columns 8 through 10

    0.7960    0.8355    0.8355    0.9090
    0.6730    0.7080    0.7080    0.7227
    0.4723    0.5714    0.5714    0.7227
    0.4723    0.5714    0.5714    0.7227
    0.3750    0.4275    0.5714    0.5714
    0.3750    0.4275    0.5714    0.5714
         0    0.2960    0.3710    0.4380
    0.2960         0    0.3710    0.4380
    0.3710    0.3710         0    0.4000
    0.4380    0.4380    0.4000         0


vaf =

    0.7210


outperm =

     1     2     3     5     4     6     7    10     9     8
\end{verbatim}

\large

\subsection{The Use of Optimal Transformations and the M-function proxmon.m}

As previously discussed, the function \verb+proxmon.m+,  provides a mono\-ton\-ically transformed proximity matrix that is close in a least-squares sense to a given input matrix.  The syntax is
\begin{verbatim}
[monproxpermut, vaf, diff] = proxmon(proxpermut,fitted)
\end{verbatim}
\noindent where \verb+PROXPERMUT+ is the input proximity matrix (which may have been subjected to an initial row/column permutation, hence the suffix `\verb+PERMUT+') and \verb+FITTED+ is a given target matrix;  the output matrix \verb+MONPROXPERMUT+ is closest to \verb+FITTED+ in a least-squares sense and obeys the order constraints obtained from each pair of entries in (the upper-triangular portion of) \verb+PROXPERMUT+ (and where the inequality constrained optimization is carried out using the Dykstra-Kaczmarz iterative projection strategy);  \verb+VAF+ indicates how much variance in \verb+MONPROXPERMUT+ can be accounted for by \verb+FITTED+; finally, \verb+DIFF+ is the value of the least-squares loss function and is (one-half)  the sum of squared differences between the entries in \verb+MONPROXPERMUT+ and \verb+FITTED+.

\smallskip

In the notation of the introduction when fitting a given order, \verb+FITTED+ would correspond to the AR matrix $\mathbf{A} = \{a_{ij}\}$; the input \verb+PROXPERMUT+ would be $\{p_{\rho^{0}(i) \rho^{0}(j)}\}$; \verb+MONPROXPERMUT+ would be $\{f(p_{\rho^{0}(i) \rho^{0}(j)})\}$, where the function $f(\cdot)$ satisfies the monotonicity constraints, i.e., if
$p_{\rho^{0}(i) \rho^{0}(j)} < p_{\rho^{0}(i') \rho^{0}(j')}$ for $1 \le i  < j \le n$ and $1 \le i' < j' \le n$, then $f(p_{\rho^{0}(i) \rho^{0}(j)}) \le f(p_{\rho^{0}(i') \rho^{0}(j')})$.  The transformed proximity matrix $\{f(p_{\rho^{0}(i) \rho^{0}(j)})\}$ minimizes the least-squares criterion (\verb+DIFF+) of
\[ \sum_{i < j} (f(p_{\rho^{0}(i) \rho^{0}(j)}) - a_{ij})^{2} , \]
over all functions $f(\cdot)$ that satisfy the monotonicity constraints.  The \verb+VAF+ is a normalization of this loss value by the sum of squared deviations of the transformed proximities from their mean:
\[ \mbox{VAF} = 1 - \frac{\sum_{i < j} (f(p_{\rho^{0}(i) \rho^{0}(j)}) - a_{ij})^{2}}{\sum_{i < j} (f(p_{\rho^{0}(i) \rho^{0}(j)}) - \bar{f})^{2}} , \]
where $\bar{f}$ denotes the mean of the off-diagonal entries in $\{f(p_{\rho^{0}(i) \rho^{0}(j)})\}$.

\smallskip


The script M-file listed below gives an application of \verb+proxmon.m+  along with finding a best-fitting AR form for our \verb+number.dat+ matrix.  First, \verb+arobfnd.m+ is invoked to obtain a  best-fitting AR matrix (\verb+find+); this is the same as found earlier based on the \verb+outperm+ of [1 2 3 5 4 6 7 9 10 8] and generating a \verb+VAF+ of .7782.  The M-file, \verb+proxmon.m+, is then used to generate the monotonically transformed proximity matrix (\verb+monproxpermut+) with \verb+VAF+ of .8323.  Given the SIOT (single-iteration-optimal-transformation) discussed in the introduction, it might now be best to fit once more an AR matrix to this now monotonically transformed proximity matrix, but then stop.  Otherwise, as seen in the output below, if the strategy is repeated cyclically (i.e., finding a fitted matrix based on the monotonically transformed proximity matrix, finding a new monotonically transformed matrix, and so on), a perfect \verb+VAF+ of 1.0 can be achieved at the expense of losing most of the detail in the transformed proximities, i.e., only five distinct values remain that correspond to the three largest and single smallest of the original proximities with \emph{all} the remaining now tied at a value of .5467. (To avoid another type of degeneracy [where all matrices would converge to zeros], the sum of squares of the fitted matrix was maintained stationary; convergence is based on observing a minimal change (less than 1.0e-010) in the \verb+VAF+.

\tiny

\begin{verbatim}
load number.dat
[find, vaf, outperm] = arobfnd(number,randperm(10),2)
[monproxpermut vaf diff] = ...
    proxmon(number(outperm,outperm),find)
sumfitsq = sum(sum(fit.^2));
prevvaf = 2;
while (abs(prevvaf-vaf) >= 1.0e-010)
   prevvaf = vaf;
   [fit vaf] = arobfit(monproxpermut,1:10);
   sumnewfitsq = sum(sum(fit.^2));
   find = sqrt(sumfitsq)*(fit/sqrt(sumnewfitsq));
    [monproxpermut, vaf, diff] = ...
      proxmon(number(outperm,outperm), find);
end

outperm
find


monproxpermut
vaf
diff

find =

  Columns 1 through 6

         0    0.4210    0.5840    0.6840    0.7090    0.7960
    0.4210         0    0.2840    0.4960    0.4960    0.5880
    0.5840    0.2840         0    0.0590    0.3835    0.4928
    0.6840    0.4960    0.0590         0    0.3835    0.3985
    0.7090    0.4960    0.3835    0.3835         0    0.3750
    0.7960    0.5880    0.4928    0.3985    0.3750         0
    0.7960    0.7357    0.4928    0.3985    0.3750    0.3750
    0.8210    0.7357    0.4928    0.4928    0.4928    0.4928
    0.8500    0.7357    0.7357    0.6830    0.4928    0.4928
    0.9090    0.7357    0.7357    0.7357    0.5920    0.4928

  Columns 7 through 10

    0.7960    0.8210    0.8500    0.9090
    0.7357    0.7357    0.7357    0.7357
    0.4928    0.4928    0.7357    0.7357
    0.3985    0.4928    0.6830    0.7357
    0.3750    0.4928    0.4928    0.5920
    0.3750    0.4928    0.4928    0.4928
         0    0.3460    0.3460    0.4253
    0.3460         0    0.3460    0.4253
    0.3460    0.3460         0    0.4253
    0.4253    0.4253    0.4253         0


vaf =

    0.7782


outperm =

     1     2     3     5     4     6     7     9    10     8


monproxpermut =

  Columns 1 through 6

         0    0.4244    0.5549    0.6840    0.7058    0.7659
    0.4244         0    0.3981    0.5908    0.4054    0.5549
    0.5549    0.3981         0    0.0590    0.4054    0.5908
    0.6840    0.5908    0.0590         0    0.4244    0.4244
    0.7058    0.4054    0.4054    0.4244         0    0.4310
    0.7659    0.5549    0.5908    0.4244    0.4310         0
    0.7058    0.7058    0.4310    0.4054    0.3981    0.4054
    0.8210    0.7058    0.4054    0.3981    0.7058    0.5908
    0.8500    0.5908    0.7659    0.6830    0.3981    0.5549
    0.9090    0.5908    0.7058    0.7058    0.5908    0.4244

  Columns 7 through 10

    0.7058    0.8210    0.8500    0.9090
    0.7058    0.7058    0.5908    0.5908
    0.4310    0.4054    0.7659    0.7058
    0.4054    0.3981    0.6830    0.7058
    0.3981    0.7058    0.3981    0.5908
    0.4054    0.5908    0.5549    0.4244
         0    0.4054    0.3981    0.4244
    0.4054         0    0.4054    0.4244
    0.3981    0.4054         0    0.4310
    0.4244    0.4244    0.4310         0


vaf =

    0.8323


diff =

    0.2075


outperm =

     1     2     3     5     4     6     7     9    10     8


find =

  Columns 1 through 6

         0    0.5467    0.5467    0.5467    0.5467    0.5467
    0.5467         0    0.5467    0.5467    0.5467    0.5467
    0.5467    0.5467         0    0.0609    0.5467    0.5467
    0.5467    0.5467    0.0609         0    0.5467    0.5467
    0.5467    0.5467    0.5467    0.5467         0    0.5467
    0.5467    0.5467    0.5467    0.5467    0.5467         0
    0.5467    0.5467    0.5467    0.5467    0.5467    0.5467
    0.8474    0.5467    0.5467    0.5467    0.5467    0.5467
    0.8774    0.5467    0.5467    0.5467    0.5467    0.5467
    0.9383    0.5467    0.5467    0.5467    0.5467    0.5467

  Columns 7 through 10

    0.5467    0.8474    0.8774    0.9383
    0.5467    0.5467    0.5467    0.5467
    0.5467    0.5467    0.5467    0.5467
    0.5467    0.5467    0.5467    0.5467
    0.5467    0.5467    0.5467    0.5467
    0.5467    0.5467    0.5467    0.5467
         0    0.5467    0.5467    0.5467
    0.5467         0    0.5467    0.5467
    0.5467    0.5467         0    0.5467
    0.5467    0.5467    0.5467         0


monproxpermut =

  Columns 1 through 6

         0    0.5467    0.5467    0.5467    0.5467    0.5467
    0.5467         0    0.5467    0.5467    0.5467    0.5467
    0.5467    0.5467         0    0.0609    0.5467    0.5467
    0.5467    0.5467    0.0609         0    0.5467    0.5467
    0.5467    0.5467    0.5467    0.5467         0    0.5467
    0.5467    0.5467    0.5467    0.5467    0.5467         0
    0.5467    0.5467    0.5467    0.5467    0.5467    0.5467
    0.8474    0.5467    0.5467    0.5467    0.5467    0.5467
    0.8774    0.5467    0.5467    0.5467    0.5467    0.5467
    0.9383    0.5467    0.5467    0.5467    0.5467    0.5467

  Columns 7 through 10

    0.5467    0.8474    0.8774    0.9383
    0.5467    0.5467    0.5467    0.5467
    0.5467    0.5467    0.5467    0.5467
    0.5467    0.5467    0.5467    0.5467
    0.5467    0.5467    0.5467    0.5467
    0.5467    0.5467    0.5467    0.5467
         0    0.5467    0.5467    0.5467
    0.5467         0    0.5467    0.5467
    0.5467    0.5467         0    0.5467
    0.5467    0.5467    0.5467         0


vaf =

    1.0000


diff =

  8.3999e-011
\end{verbatim}

\large

\subsection{Graphically Representing SAR Structures}

The use of the very general form of representation offered by an AR matrix without the imposition of any further restrictions has one annoying interpretive difficulty.  Specifically, it is usually necessary to interpret the fitted structures directly (and enumeratively) through a set of  subsets or clusters that are all defined by objects contiguous in a specific object ordering; each such subset has an attached diameter that reflects its maximum within-class fitted value.  More pointedly, it is generally \emph{not} possible to use a more convenient graph-theoretic structure and the lengths of paths between objects in such a graph to represent visually a fitted AR matrix; this situation contrasts with opportunities resulting when the approximation matrix is more restricted and defined, say, by an ultrametric or an additive tree, or by a (linear or circular) unidimensional scaling.

\smallskip


As noted in the introduction, the imposition of SAR conditions allows a representation of the fitted values in a (least-squares) SAR approximating matrix as lengths of paths in a graph, although this graph will not generally have the simplified form of a tree.  A discussion of these latter SAR constraints is not new here, and several (theoretical) presentations of their usefulness exist in the literature (for example, see Critchley and Fichet, 1994; Critchley, 1994; Durand and Fichet, 1988; Mirkin, 1996, Chapter 7).   Here, we give the example based on the number data from Hubert, Arabie, and Meulman (1998)    for interpretative convenience. The latter data were transformed (in that reference) to a standard deviation of 1.0 and a mean of 4.0; thus, the numbers within the fitted matrices will differ from the examples given earlier. Approximating AR and SAR forms for the transformed number proximity data are given in the upper and lower-triangular portions, respectively, of the matrix in Table X. For convenience below, we will denote the upper-triangular AR matrix by $\mathbf{A}_{ut}$ and the lower-triangular SAR matrix by $\mathbf{A}_{lt}$.



\begin{table}


\caption{Order-constrained least-squares approximations to the
digit proximity data of Shepard \emph{et al}.\ (1975);
the upper-triangular portion is anti-Robinson and the
lower-triangular portion is strongly-anti-Robinson}

\bigskip

\begin{center}

\begin{tabular}{ccccccccccc}

digit & 0 & 1 & 2 & 4 & 3 & 5 & 6 & 8 & 9 & 7 \\ 0 & x & 3.41 &
4.21 & 4.70 & 4.83 & 5.25 & 5.25 & 5.38 & 5.52 & 5.81 \\ 1 & 3.41
& x & 2.73 & 3.78 & 3.78 & 4.23 & 4.96 & 4.96 & 4.96 & 4.96 \\ 2
& 4.21 & 2.73 & x & 1.63 & 3.22 & 3.76 & 3.76 & 3.76 & 4.96 &
4.96 \\ 4 & 4.76 & 3.78 & 1.63 & x & 3.22 & 3.30 & 3.30 & 3.76 &
4.70 & 4.96 \\ 3 & 4.76 & 3.78 & 3.22 & 3.22 & x & 3.18 & 3.18 &
3.76 & 3.76 & 4.25 \\ 5 & 5.25 & 4.59 & 3.53 & 3.53 & 3.18 & x &
3.18 & 3.76 & 3.76 & 3.76 \\ 6 & 5.25 & 4.59 & 3.53 & 3.53 & 3.18
& 3.18 & x & 3.04 & 3.04 & 3.43 \\ 8 & 5.57 & 4.96 & 4.18 & 4.18
& 4.18 & 4.18 & 3.04 & x & 3.04 & 3.43 \\ 9 & 5.57 & 4.96 & 4.18
& 4.18 & 4.18 & 4.18 & 3.04 & 3.04 & x & 3.43 \\ 7 & 5.57 & 4.96
& 4.18 & 4.18 & 4.18 & 4.18 & 3.43 & 3.43 & 3.43 & x \\

\end{tabular}

\end{center}

\end{table}

\smallskip

The $10(10-1)/2 =
45$ subsets defined by objects contiguous in the object ordering used to
display the upper-triangular portion of Table 3 are listed
in Table 4 according to increasing diameter values.  For purposes of our later discussion, 22 of the subsets are given in italics to indicate that they are proper subsets of another listed subset having the same diameter.
Substantively, the dominant patterning of the entries in
$\mathbf{A}_{ut}$ appears to reflect (primarily) digit
magnitude except for the placement of digit 4 next to 2, and
digit 7 being located in the last position.  Both  these latter
deviations from an interpretation strictly according to digit
magnitude show some of the salient structural properties of
the digits.  For example, the digit pair (2,4) has the absolute
smallest dissimilarity in the data; besides being relatively
close in magnitude, there are the possible (although redundant)
similarity bases that
$2 + 2=4$, $2 \times 2 = 4$, 4 is a power of 2, and both 2 and 4
are even numbers.  Similarly, the placement of the digit 7 in the
last position results from the salience of the triple
$\{6,8,9\}$, which is the third to emerge according to its
diameter.
In addition to these three digits all being relatively close in
magnitude, 6 and 8 are both even numbers, 6 and 9 are multiples
of 3, and 8 is directly adjacent in size to 9.  The three
original dissimilarities within the set $\{6,8,9\}$ are all
smaller than the dissimilarities digit 7 has to \emph{any} other
digit.

\begin{table}



\caption{The 45 subsets listed according to increasing diameter values that are contiguous in the object ordering used to display the upper-triangular portion of Table 3.  The 22 subsets given in italics are redundant in the sense that they are proper subsets of another listed subset with the same diameter.}

\bigskip


\begin{center}

\begin{tabular}{lc}




\emph{subset} & \emph{diameter}\\
 \{2,4\} & 1.63 \\
    \{1,2\} & 2.73 \\
\emph{\{6,8\}},\emph{\{8,9\}},\{6,8,9\} & 3.04 \\
\emph{\{3,5\}},\emph{\{5,6\}},\{3,5,6\} & 3.18 \\
\emph{\{4,3\}},\{2,4,3\} & 3.22 \\
\emph{\{4,3,5\}},\{4,3,5,6\} & 3.30 \\
\{0,1\} & 3.41 \\
\emph{\{9,7\}},\emph{\{8,9,7\}},\{6,8,9,7\} & 3.43 \\
\emph{\{5,6,8\}},\emph{\{5,6,8,9\}},\{5,6,8,9,7\} & 3.76 \\
\emph{\{3,5,6,8\}},\{3,5,6,8,9\} & 3.76 \\
\emph{\{4,3,5,6,8\}},\emph{\{2,4,3,5\}},\emph{\{2,4,3,5,6\}},\{2,4,3,5,6,8\} \hspace{3ex} & 3.76 \\
\emph{\{1,2,4\}},\{1,2,4,3\} & 3.78 \\

  \{0,1,2\}& 4.21 \\

  \{1,2,4,3,5\} & 4.23 \\
 \{3,5,6,8,9,7\} & 4.25 \\

\{0,1,2,4\},\{4,3,5,6,8,9\} & 4.70 \\

\{0,1,2,4,3\} & 4.83 \\

\emph{\{1,2,4,3,5,6\}},\emph{\{1,2,4,3,5,6,8\}} & 4.96 \\
\emph{\{1,2,4,3,5,6,8,9\}},\emph{\{2,4,3,5,6,8,9\}} & 4.96 \\
\emph{\{2,4,3,5,6,8,9,7\}},\emph{\{4,3,5,6,8,9,7\}} & 4.96 \\


\{1,2,4,3,5,6,8,9,7\} & 4.96 \\
\emph{\{0,1,2,4,3,5\}},\{0,1,2,4,3,5,6\} & 5.25 \\
  \{0,1,2,4,3,5,6,8\} & 5.38 \\

\{0,1,2,4,3,5,6,8,9\} & 5.52 \\
\{0,1,2,4,3,5,6,8,9,7\} & 5.81 \\



\end{tabular}

\end{center}

\end{table}


\smallskip

Given just the collection of subsets $S_{1},\ldots,S_{M}$ listed in Table 4 and their
associated diameters, it is possible  (trivially) to reconstruct the original
approximating matrix $\mathbf{A}_{ut}$ by identifying for each object
pair the smallest diameter for a subset that contains that pair. (Explicitly, the smallest diameter for a subset that contains an object pair is equal to the value in $\mathbf{A}_{ut}$ associated with that pair, and the subset itself includes that object pair and all objects in between in the ordering that is used to display the AR form for $\mathbf{A}_{ut}$.)
This type of reconstruction is generally possible for any matrix
that can be row/column reordered to an AR form through the
collection of subsets $S_{1},\ldots,S_{M}$ and their diameters identified by
increasing a threshold variable from the smallest fitted value.  In fact, even if all the italicized subsets were removed (that are proper subsets of another having the same diameter), exactly the same reconstruction could be carried out because the italicized subsets are redundant with respect to identifying for each object pair the smallest diameter for a subset that contains the pair.

\smallskip

Without imposing
further  restrictions on the approximating matrix other than just being AR, a
more convenient representation using a graph and path lengths in such a
graph is generally not possible.  We will select two small
(AR)  submatrices from the upper-triangular portion of Table 3 to
make this  point more convincingly, and in the process indicate by example how a graph representation is to be constructed and why further restrictions on the approximating matrix may be necessary to carry out the task.

\smallskip


First, consider the fitted values for the first four placed
digits, 0, 1, 2, and 4 in Figure 1(a), for which the desired type of graphical
representation \emph{is} possible without imposing any further constraints.  This AR submatrix is given in
Figure 1(a) along with the six corresponding subsets of contiguous objects and their
diameters,
and a graphical representation for the structure.  The
latter consists of four  nodes corresponding to the
original four objects that we represent by open circles (referred to as ``terminal'' nodes), plus six  nodes represented by solid circles that denote the
six subsets in the given listing (referred to as ``internal'' nodes).  Based on this graph and the
internal node heights provided by the calibrated scale on the
left, a fitted
value in the submatrix between any two
terminal nodes can be obtained as one-half the length of the
minimum path from one of the terminal nodes up to an internal
node and back down to the other terminal node.  All horizontal
line segments are used here for display convenience only and
are not actually assumed to contribute to the length of any path.  Thus,
if we changed the vertical scaling by a multiplier of 1/2, each
of the fitted values in the submatrix would be exactly the length
of the minimum path between two terminal nodes, that proceeded
upward from one such node to an internal node and then back down
to the other.  We might also note that from the topmost internal
node, all paths down to the terminal nodes have exactly the same
length; i.e., there is an internal node equidistant from all
terminal nodes.

\smallskip

Now, consider the fitted values for the four objects placed
respectively at
the third through sixth positions: 2, 4, 3, and 5, given in Figure 1(b) along with the corresponding
subsets of contiguous objects and their diameters (excluding the redundant subset \{4,3\} which is a proper subset of \{2,4,3\} having the same diameter),
and the beginnings of a graphical representation for its
structure.  There is a difficulty encountered, however, in
defining a graph that would be completely
consistent with all the fitted values in the $4 \times 4$
submatrix; we indicate this anomaly by the dashed vertical and
horizontal
lines.  If an internal node were to be placed at the level of
3.30 to represent the cluster $\{4,3,5\}$, by implication the
fitted value for the digit pair (2,5) should also be 3.30 (and
not its current value of 3.76).  Because digit 3 was
``joined'' to \emph{both} 2 and 4 at the threshold level 3.22,
and thus, there are two fitted values tied at 3.22, a consistent
graphical representation would be possible only if the fitted
values for the pairs (2,5) and (4,5) were equal.  \emph{This last
observation, that when some fitted values are tied in an
approximating matrix $\mathbf{A}_{ut}$, others must also be
tied to allow for the construction of a consistent graphical
representation, is the motivating basis for considering an
additional set of SAR constraints}.


\smallskip

\setlength{\unitlength}{1.5mm}

\begin{figure}

\begin{center}

\vspace{10mm}

\begin{picture}(70.0,85.0)(0.0,-12.0)



\put(30.0,-10.0){\circle{1}} \put(35.0,-10.0){\circle{1}}
\put(40.0,-10.0){\circle{1}} \put(45.0,-10.0){\circle{1}}
\put(32.5,6.3){\circle*{1}} \put(37.0,22.2){\circle*{1}}
\put(42.5,21.8){\circle*{1}}



\put(30.5,-10.0){\line(0,1){16.3}}
\put(34.5,-10.0){\line(0,1){16.3}}
\put(39.5,-10.0){\line(0,1){32.2}}
\put(40.5,-10.0){\line(0,1){31.8}}
\put(44.5,-10.0){\line(0,1){31.8}}
\put(33.0,6.3){\line(0,1){15.9}}





\put(30.5,6.3){\line(1,0){4.0}} \put(40.5,21.8){\line(1,0){4.0}}
\put(33.0,22.2){\line(1,0){6.5}}

\put(25.0,-10.0){\line(0,1){35.0}}

\put(25.0,6.3){\line(1,0){2.0}} \put(25.0,21.8){\line(1,0){2.0}}
\put(25.0,22.2){\line(1,0){2.0}} \put(25.0,23.0){\line(1,0){2.0}}

\put(23.0,6.3){\makebox(0,0)[br]{1.63}}
\put(22.0,20.0){\makebox(0,0)[br]{3.18}}
\put(22.0,22.){\makebox(0,0)[br]{3.22}}
\put(22.0,24.0){\makebox(0,0)[br]{3.30}}

\put(22.0,22.0){\vector(1,0){2}}
\put(22,20){\vector(2,1){2}}
\put(22,24){\vector(2,-1){2}}

\curvedashes[1.0pt]{0,1,2}


%\put(37.5,22.2){\curve(37.5,22.2, 37.5,23.0)}
%\put(37.5,23.0){\curve(37.5,23.0, 42.0,23.0)}
%\put(42.0,23.0){\curve(42.0,23.0, 42.0,22.2)}
\put(37.5,22.2){\curve(0,0, 0,.8)}
\put(42.0,22.2){\curve(0,0, 0,.8)}
\put(37.5,23.0){\curve(0,0, 4.5,0)}
 \curvedashes{}
\put(50.0,20.0){\makebox(0,0)[bl]{\underline{\emph{subset}}}}
\put(50.0,17.0){\makebox(0,0)[bl]{\{2,4\}}}
\put(50.0,14.0){\makebox(0,0)[bl]{\{3,5\}}}
\put(50.0,11.0){\makebox(0,0)[bl]{\{2,4,3\}}}
\put(50.0,8.0){\makebox(0,0)[bl]{\{4,3,5\}}}
\put(50.0,5.0){\makebox(0,0)[bl]{\{2,4,3,5\}}}

\put(60.0,20.0){\makebox(0,0)[b]{\hspace{1ex}%
\underline{\emph{diam}}}}
\put(60.0,17.0){\makebox(0,0)[bl]{1.63}}
\put(60.0,14.0){\makebox(0,0)[bl]{3.18}}
\put(60.0,11.0){\makebox(0,0)[bl]{3.22}}
\put(60.0,8.0){\makebox(0,0)[bl]{3.30}}
\put(60.0,5.0){\makebox(0,0)[bl]{3.76}}

\put(4.0,21.0){\makebox(0,0)[bl]{2}}
\put(8.0,21.0){\makebox(0,0)[bl]{4}}
\put(12.0,21.0){\makebox(0,0)[bl]{3}}
\put(16.0,21.0){\makebox(0,0)[bl]{5}}

\put(1.0,18.0){\makebox(0,0)[br]{2}}
\put(1.0,15.0){\makebox(0,0)[br]{4}}
\put(1.0,12.0){\makebox(0,0)[br]{3}}
\put(1.0,9.0){\makebox(0,0)[br]{5}}

\put(2.0,7.0){\line(0,1){15.0}} \put(0.0,20.0){\line(1,0){18.0}}

\put(4.0,18.0){\makebox(0,0)[b]{x}}
\put(8.0,18.0){\makebox(0,0)[b]{1.63\hspace{1em}}}
\put(8.0,15.0){\makebox(0,0)[b]{x}}
\put(12.0,18.0){\makebox(0,0)[b]{3.22\hspace{1ex}}}
\put(12.0,15.0){\makebox(0,0)[b]{3.22\hspace{1ex}}}
\put(12.0,12.0){\makebox(0,0)[b]{x}}
\put(16.0,18.0){\makebox(0,0)[b]{3.76}}
\put(16.0,15.0){\makebox(0,0)[b]{3.30}}
\put(16.0,12.0){\makebox(0,0)[b]{3.18}}
\put(16.0,9.0){\makebox(0,0)[b]{x}}

\put(0.0,26.0){\makebox(0,0){(b)}}


\put(30.0,30.0){\circle{1}} \put(35.0,30.0){\circle{1}}
\put(40.0,30.0){\circle{1}} \put(45.0,30.0){\circle{1}}
\put(32.5,64.1){\circle*{1}} \put(37.5,57.3){\circle*{1}}
\put(42.5,46.3){\circle*{1}} \put(35.0,72.1){\circle*{1}}
\put(40.0,67.8){\circle*{1}} \put(37.5,77.0){\circle*{1}}


\put(30.5,30.0){\line(0,1){34.1}}
\put(34.5,30.0){\line(0,1){34.1}}
\put(35.5,30,0){\line(0,1){27.3}}
\put(39.5,30.0){\line(0,1){27.3}}
\put(40.5,30.0){\line(0,1){16.3}}
\put(44.5,30.0){\line(0,1){16.3}}
\put(33.0,64.1){\line(0,1){8.0}}
\put(37.0,57.3){\line(0,1){14.8}}
\put(38.0,57.3){\line(0,1){10.5}}
\put(42.0,46.3){\line(0,1){21.5}}
\put(35.5,72.1){\line(0,1){4.9}} \put(39.5,67.8){\line(0,1){9.2}}





\put(30.5,64.1){\line(1,0){4.0}} \put(35.5,57.3){\line(1,0){4.0}}
\put(40.5,46.3){\line(1,0){4.0}} \put(33.0,72.1){\line(1,0){4.0}}
\put(38.0,67.8){\line(1,0){4.0}} \put(35.5,77.0){\line(1,0){4.0}}






\put(25.0,30.0){\line(0,1){49.0}}

\put(25.0,46.3){\line(1,0){2.0}} \put(25.0,57.3){\line(1,0){2.0}}
\put(25.0,64.1){\line(1,0){2.0}} \put(25.0,67.8){\line(1,0){2.0}}
\put(25.0,72.1){\line(1,0){2.0}} \put(25.0,77.0){\line(1,0){2.0}}



\put(23.0,46.3){\makebox(0,0)[br]{1.63}}
\put(23.0,57.3){\makebox(0,0)[br]{2.73}}
\put(23.0,64.1){\makebox(0,0)[br]{3.41}}
\put(23.0,67.8){\makebox(0,0)[br]{3.78}}
\put(23.0,72.1){\makebox(0,0)[br]{4.21}}
\put(23.0,77.0){\makebox(0,0)[br]{4.70}}


\put(50.0,60.0){\makebox(0,0)[bl]{\underline{\emph{subset}}}}
\put(50.0,57.0){\makebox(0,0)[bl]{\{2,4\}}}
\put(50.0,54.0){\makebox(0,0)[bl]{\{1,2\}}}
\put(50.0,51.0){\makebox(0,0)[bl]{\{0,1\}}}
\put(50.0,48.0){\makebox(0,0)[bl]{\{1,2,4\}}}
\put(50.0,45.0){\makebox(0,0)[bl]{\{0,1,2\}}}
\put(50.0,42.0){\makebox(0,0)[bl]{\{0,1,2,4\}}}





\put(60.0,60.0){\makebox(0,0)[b]{\hspace{1ex}%
\underline{\emph{diam}}}}
\put(60.0,57.0){\makebox(0,0)[bl]{1.63}}
\put(60.0,54.0){\makebox(0,0)[bl]{2.73}}
\put(60.0,51.0){\makebox(0,0)[bl]{3.41}}
\put(60.0,48.0){\makebox(0,0)[bl]{3.78}}
\put(60.0,45.0){\makebox(0,0)[bl]{4.21}}
\put(60.0,42.0){\makebox(0,0)[bl]{4.70}}

\put(4.0,61.0){\makebox(0,0)[bl]{0}}
\put(8.0,61.0){\makebox(0,0)[bl]{1}}
\put(12.0,61.0){\makebox(0,0)[bl]{2}}
\put(16.0,61.0){\makebox(0,0)[bl]{4}}

\put(1.0,58.0){\makebox(0,0)[br]{0}}
\put(1.0,55.0){\makebox(0,0)[br]{1}}
\put(1.0,52.0){\makebox(0,0)[br]{2}}
\put(1.0,49.0){\makebox(0,0)[br]{4}}

\put(2.0,47.0){\line(0,1){15.0}} \put(0.0,60.0){\line(1,0){18.0}}

\put(4.0,58.0){\makebox(0,0)[b]{x}}
\put(8.0,58.0){\makebox(0,0)[b]{3.41\hspace{1em}}}
\put(8.0,55.0){\makebox(0,0)[b]{x}}
\put(12.0,58.0){\makebox(0,0)[b]{4.21\hspace{1ex}}}
\put(12.0,55.0){\makebox(0,0)[b]{2.73\hspace{1ex}}}
\put(12.0,52.0){\makebox(0,0)[b]{x}}
\put(16.0,58.0){\makebox(0,0)[b]{4.70}}
\put(16.0,55.0){\makebox(0,0)[b]{3.78}}
\put(16.0,52.0){\makebox(0,0)[b]{1.63\hspace{1ex}}}
\put(16.0,49.0){\makebox(0,0)[b]{x}}

\put(0.0,79.0){\makebox(0,0){(a)}}

\put(30.0,29.0){\makebox(0,0)[t]{0}}
\put(35.0,29.0){\makebox(0,0)[t]{1}}
\put(40.0,29.0){\makebox(0,0)[t]{2}}
\put(45.0,29.0){\makebox(0,0)[t]{4}}
\put(30.0,-11.0){\makebox(0,0)[t]{2}}
\put(35.0,-11.0){\makebox(0,0)[t]{4}}
\put(40.0,-11.0){\makebox(0,0)[t]{3}}
\put(45.0,-11.0){\makebox(0,0)[t]{5}}

\end{picture}


\caption{Two  $4 \times 4$ submatrices and the object subsets
they induce, taken from the anti-Robinson matrix in the
upper-triangular portion of Table 3.  For (a), a graphical
representation of the fitted values is possible; for (b), the
anomaly  indicated by the dashed lines prevents a consistent
graphical representation from being constructed.}

\end{center}


\end{figure}






When a graphical representation that permits their reconstruction
through path
lengths is
desired for the collection
of fitted values in an approximating matrix $\mathbf{A}$,
the small
illustration just provided serves as justification for imposing a
stricter collection of constraints on the approximating matrix
than just being row/column reorderable to an AR form.  In
particular, the additional restriction will be imposed that the
approximating matrix $\mathbf{A}$ is row/column reorderable to
one that is SAR, which will eliminate the type of graphical
anomaly present in Figure 1(b).

\smallskip

For the SAR approximation given in the lower-triangular portion of Table 3, there are now only fourteen (nonredundant) subsets identifiable by increasing a
threshold variable from the smallest fitted value; these are listed in Table 5 along with
their
diameters.  The imposition of the more restrictive SAR constraints allows the
graphical representation given in Figure 2.  Although we might
not change our substantive comments about
the approximating matrix (i.e., mostly digit
magnitude with some structural characteristics for the subsets
$\{2,4\}$ and $\{6,8,9\}$), a graphical
representation makes these same observations visually clearer.



\begin{table}


\caption{The fourteen (nonredundant) subsets  listed according to increasing diameter values are contiguous in the linear object ordering used to display the lower-triangular SAR portion of Table 3.}




\bigskip

\begin{center}

\begin{tabular}{lcclc}

\emph{subset} & \emph{diameter} & \hspace{3ex} & \emph{subset} &
\emph{diameter}
\\ \{2,4\} & 1.63 & \hspace{3ex} & \{2,4,3,5,6\} & 3.53 \\
\{1,2\} & 2.73 &
\hspace{3ex} &
\{1,2,4,3\} & 3.78 \\ \{6,8,9\} & 3.04 & \hspace{3ex} &
\{2,4,3,5,6,8,9,7\} &
4.18 \\ \{3,5,6\} & 3.18 & \hspace{3ex} & \{0,1,2\} & 4.21 \\
\{2,4,3\} & 3.22 &
 \hspace{3ex} & \{0,1,2,4,3\} & 4.76 \\ \{0,1\} & 3.41 &
\hspace{3ex} & \{0,1,2,4,3,5,6\}
&
5.25
\\ \{6,8,9,7\} & 3.43 & \hspace{3ex} & \{0,1,2,4,3,5,6,8,9,7\} &
5.57 \\

\end{tabular}

\end{center}


\end{table}


\setlength{\unitlength}{1.00mm}

\begin{figure}





\begin{picture}(125,65)(-10,-15)

\put(10,-10){\circle{2}}
\put(20,-10){\circle{2}}
\put(30,-10){\circle{2}}
 \put(40,-10){\circle{2}}
\put(50,-10){\circle{2}}
 \put(60,-10){\circle{2}}
\put(70,-10){\circle{2}}
 \put(80,-10){\circle{2}}
\put(90,-10){\circle{2}}
 \put(100,-10){\circle{2}}
\put(35,6.3){\circle*{2}}
 \put(25,17.3){\circle*{2}}
\put(15,24.1){\circle*{2}}
\put(42.5,22.3){\circle*{2}}
\put(60,21.8){\circle*{2}}
 \put(80,20.4){\circle*{2}}
\put(20,32.1){\circle*{2}}
\put(33.8,27.8){\circle*{2}}
\put(51.3,25.3){\circle*{2}}
 \put(90,24.3){\circle*{2}}
\put(26.9,32,4){\circle*{2}}
 \put(70.7,31.8){\circle*{2}}
\put(39.3,42.5){\circle*{2}}
 \put(55,45.7){\circle*{2}}

\put(11,-10){\line(0,1){34.1}}
 \put(19,-10){\line(0,1){34.1}}
\put(21,-10){\line(0,1){27.3}}
\put(29,-10){\line(0,1){27.3}}
\put(31,-10){\line(0,1){16.3}}
 \put(39,-10){\line(0,1){16.3}}
\put(49,-10){\line(0,1){32.3}}
 \put(51,-10){\line(0,1){31.8}}
\put(60,-9){\line(0,1){30.8}}
 \put(69,-10){\line(0,1){31.8}}
\put(71,-10){\line(0,1){30.4}}
 \put(80,-9){\line(0,1){29.4}}
\put(89,-10){\line(0,1){30.4}}
\put(99,-10){\line(0,1){34.3}}
\put(16,24.1){\line(0,1){8.0}}
\put(24,17.3){\line(0,1){14.8}}
\put(26,17.3){\line(0,1){10.5}}
 \put(36,6.3){\line(0,1){16.0}}
\put(41.5,22.3){\line(0,1){5.5}}
 \put(43.5,22.3){\line(0,1){3.0}}
\put(59,21.8){\line(0,1){3.5}}
\put(81,20.4){\line(0,1){3.9}}
\put(21,32.1){\line(0,1){.3}}
 \put(32.8,27.8){\line(0,1){4.6}}
\put(50.3,25.3){\line(0,1){17.2}}
 \put(52.3,25.3){\line(0,1){6.5}}
\put(89,24.3){\line(0,1){7.5}}
 \put(69.7,31.8){\line(0,1){13.9}}
\put(27.9,32.4){\line(0,1){10.1}}
 \put(40.3,42.5){\line(0,1){3.2}}

\put(31,6.3){\line(1,0){8.0}}
 \put(21,17.3){\line(1,0){8.0}}
\put(11,24.1){\line(1,0){8.0}}
 \put(36,22.3){\line(1,0){13.0}}
\put(51,21.8){\line(1,0){18.0}}
\put(71,20.4){\line(1,0){18.0}}
\put(16,32.1){\line(1,0){8.0}}
 \put(26,27.8){\line(1,0){15.5}}
\put(43.5,25.3){\line(1,0){15.5}}
 \put(81,24.3){\line(1,0){18.0}}
\put(21,32.4){\line(1,0){11.8}}
 \put(52.3,31.8){\line(1,0){36.7}}
\put(27.9,42.5){\line(1,0){22.4}}
 \put(40.3,45.7){\line(1,0){29.4}}

\put(10,-12){\makebox(0,0)[t]{0}}
\put(20,-12){\makebox(0,0)[t]{1}}
\put(30,-12){\makebox(0,0)[t]{2}}
\put(40,-12){\makebox(0,0)[t]{4}}
\put(50,-12){\makebox(0,0)[t]{3}}
\put(60,-12){\makebox(0,0)[t]{5}}
\put(70,-12){\makebox(0,0)[t]{6}}
\put(80,-12){\makebox(0,0)[t]{8}}
\put(90,-12){\makebox(0,0)[t]{9}}
\put(100,-12){\makebox(0,0)[t]{7}}

\put(0.0,6.3){\line(1,0){4.0}} \put(0.0,17.3){\line(1,0){4.0}}
\put(0.0,20.4){\line(1,0){4.0}} \put(0.0,21.8){\line(1,0){4.0}}
\put(0.0,22.3){\line(1,0){4.0}} \put(0.0,24.1){\line(1,0){4.0}}
\put(0.0,24.3){\line(1,0){4.0}} \put(0.0,25.3){\line(1,0){4.0}}
\put(0.0,27.8){\line(1,0){4.0}} \put(0.0,31.8){\line(1,0){4.0}}
\put(0.0,32.1){\line(1,0){4.0}} \put(0.0,42.5){\line(1,0){4.0}}
\put(0.0,45.7){\line(1,0){4.0}} \put(0.0,32.4){\line(1,0){4.0}}


\put(-4.0,5.0){\makebox(0,0)[br]{1.5}}
\put(-4.0,10.0){\makebox(0,0)[br]{2.0}}
\put(-4.0,15.0){\makebox(0,0)[br]{2.5}}
\put(-4.0,20.0){\makebox(0,0)[br]{3.0}}
\put(-4.0,25.0){\makebox(0,0)[br]{3.5}}
\put(-4.0,30.0){\makebox(0,0)[br]{4.0}}
\put(-4.0,35.0){\makebox(0,0)[br]{4.5}}
\put(-4.0,40.0){\makebox(0,0)[br]{5.0}}
\put(-4.0,45.0){\makebox(0,0)[br]{5.5}}

\put(-4.0,5.0){\vector(1,0){3}} \put(-4.0,10.0){\vector(1,0){3}}
\put(-4.0,15.0){\vector(1,0){3}} \put(-4.0,20.0){\vector(1,0){3}}
\put(-4.0,25.0){\vector(1,0){3}} \put(-4.0,30.0){\vector(1,0){3}}
\put(-4.0,35.0){\vector(1,0){3}} \put(-4.0,40.0){\vector(1,0){3}}
\put(-4.0,45.0){\vector(1,0){3}}

\put(0.0,-10.0){\line(0,1){60.0}}

\end{picture}

\caption{A graphical representation for the fitted values given
by the strongly-anti-Robinson matrix in the lower-triangular
portion of Table 3.}



\end{figure}







\subsection{Representation Through Multiple (Strongly) AR Matrices}

The representation of a proximity matrix by a single anti-Robinson structure extends easily to the additive use of multiple matrices.  The M-function, \verb+biarobfnd.m+, fits the sum of two anti-Robinson matrices using iterative projection to a symmetric proximity matrix in the $L_{2}$-norm based on permutations
 identified through the use of iterative quadratic assignment.  The usage syntax is
 \begin{verbatim}
[find,vaf,targone,targtwo,outpermone,outpermtwo] = ...
biarobfnd(prox,inperm,kblock)
\end{verbatim}
 \noindent where, as before, \verb+PROX+ is the input proximity matrix ($n \times n$ with a zero main diagonal
 and a dissimilarity interpretation);
 \verb+INPERM+ is a given starting permutation of the first $n$ integers;
 \verb+FIND+ is the least-squares optimal matrix (with variance-accounted-for
 of \verb+VAF+) to \verb+PROX+ and is the sum of the two anti-Robinson matrices
 \verb+TARGONE+ and \verb+TARGTWO+ based on the two row and column
 object orderings given by the ending permutations \verb+OUTPERMONE+
 and \verb+OUTPERMTWO+. As before, \verb+KBLOCK+ defines the block size in the use of the
 iterative quadratic assignment routine.

 \smallskip

In the example below, the two resulting AR forms are very clearly interpretable as number magnitude and digit structural properties; the \verb+VAF+ is, in effect, 100\%.


\tiny


\begin{verbatim}
load number.dat
[find,vaf,targone,targtwo,outpermone,outpermtwo] = ...
biarobfnd(number,1:10,1)


find =

  Columns 1 through 6

         0    0.4209    0.5840    0.7090    0.6840    0.8040
    0.4209         0    0.2840    0.3460    0.6460    0.5880
    0.5840    0.2840         0    0.3540    0.0588    0.6702
    0.7090    0.3460    0.3540         0    0.4130    0.4290
    0.6840    0.6460    0.0588    0.4130         0    0.4094
    0.8040    0.5880    0.6702    0.4290    0.4094         0
    0.7865    0.7568    0.4225    0.3000    0.3880    0.3960
    0.9107    0.6300    0.7960    0.5920    0.7420    0.4000
    0.8210    0.7975    0.3672    0.7975    0.2460    0.6714
    0.8500    0.6250    0.8080    0.2630    0.6829    0.5920

  Columns 7 through 10

    0.7865    0.9107    0.8210    0.8500
    0.7568    0.6300    0.7975    0.6250
    0.4225    0.7960    0.3672    0.8080
    0.3000    0.5920    0.7975    0.2630
    0.3880    0.7420    0.2460    0.6829
    0.3960    0.4000    0.6714    0.5920
         0    0.4169    0.3499    0.2960
    0.4169         0    0.4000    0.4587
    0.3499    0.4000         0    0.3922
    0.2960    0.4587    0.3922         0


vaf =

    0.9999


targone =

  Columns 1 through 6

         0    0.3406    0.6710    0.6926    0.6956    0.6956
    0.3406         0    0.2018    0.5421    0.5423    0.5880
    0.6710    0.2018         0    0.3333    0.3680    0.4662
    0.6926    0.5421    0.3333         0    0.3093    0.3206
    0.6956    0.5423    0.3680    0.3093         0    0.2055
    0.6956    0.5880    0.4662    0.3206    0.2055         0
    0.8303    0.6764    0.4662    0.3779    0.3779    0.2876
    0.8303    0.6764    0.6764    0.6764    0.6383    0.4675
    0.8303    0.7511    0.7138    0.6764    0.6383    0.4745
    0.8611    0.7943    0.7943    0.6764    0.6690    0.4836

  Columns 7 through 10

    0.8303    0.8303    0.8303    0.8611
    0.6764    0.6764    0.7511    0.7943
    0.4662    0.6764    0.7138    0.7943
    0.3779    0.6764    0.6764    0.6764
    0.3779    0.6383    0.6383    0.6690
    0.2876    0.4675    0.4745    0.4836
         0    0.3360    0.3366    0.3849
    0.3360         0    0.2243    0.3783
    0.3366    0.2243         0    0.3783
    0.3849    0.3783    0.3783         0


targtwo =

  Columns 1 through 6

         0   -0.3923   -0.3092   -0.0093    0.0139    0.0139
   -0.3923         0   -0.3092   -0.0116    0.0101    0.0139
   -0.3092   -0.3092         0   -0.0870   -0.0438    0.0137
   -0.0093   -0.0116   -0.0870         0   -0.0438   -0.0111
    0.0139    0.0101   -0.0438   -0.0438         0   -0.0889
    0.0139    0.0139    0.0137   -0.0111   -0.0889         0
    0.1211    0.1037    0.0207    0.0164   -0.0779   -0.4134
    0.1211    0.1037    0.0822    0.0804    0.0804   -0.1693
    0.1757    0.1037    0.0822    0.0804    0.0804    0.0804
    0.2039    0.2039    0.2039    0.1084    0.1084    0.1084

  Columns 7 through 10

    0.1211    0.1211    0.1211    0.1757    0.2039
    0.1037    0.1037    0.1037    0.1037    0.2039
    0.0207    0.0207    0.0822    0.0822    0.2039
    0.0164    0.0164    0.0804    0.0804    0.1084
   -0.0779   -0.0779    0.0804    0.0804    0.1084
   -0.4134   -0.4134   -0.1693    0.0804    0.1084
         0         0   -0.1961   -0.0844    0.1084
   -0.1961   -0.1961         0   -0.1211         0
   -0.0844   -0.0844   -0.1211         0   -0.0745
    0.1084    0.1084         0   -0.0745         0


outpermone =

     1     2     3     4     5     6     7     9     8    10


outpermtwo =

     9     5     3     1     7    10     4     2     8     6
\end{verbatim}

\large

For finding multiple SAR forms,  \verb+bisarobfnd.m+ has usage syntax
\begin{verbatim}
[find,vaf,targone,targtwo,outpermone,outpermtwo] = ...
bisarobfnd(prox,inperm,kblock)
\end{verbatim}
\noindent with all the various terms the same as for \verb+biarobfnd.m+ but now for strongly AR (SAR) structures.  The example below finds essentially the same representation as above (involving digit magnitude and structure) with a slight drop in the \verb+VAF+ to  99.06\%.

\tiny

\begin{verbatim}
[find,vaf,targone,targtwo,outpermone,outpermtwo] = ...
bisarobfnd(number,randperm(10),1)

find =

  Columns 1 through 6

         0    0.4210    0.5840    0.7095    0.6838    0.8519
    0.4210         0    0.2840    0.3460    0.6461    0.5892
    0.5840    0.2840         0    0.3541    0.0590    0.6090
    0.7095    0.3460    0.3541         0    0.4131    0.4278
    0.6838    0.6461    0.0590    0.4131         0    0.4090
    0.8519    0.5892    0.6090    0.4278    0.4090         0
    0.7260    0.7565    0.4830    0.3005    0.3882    0.3960
    0.8998    0.6153    0.8059    0.6067    0.7286    0.4000
    0.8208    0.8246    0.3670    0.7893    0.2460    0.6711
    0.8736    0.6250    0.7797    0.2630    0.6965    0.5920

  Columns 7 through 10

    0.7260    0.8998    0.8208    0.8736
    0.7565    0.6153    0.8246    0.6250
    0.4830    0.8059    0.3670    0.7797
    0.3005    0.6067    0.7893    0.2630
    0.3882    0.7286    0.2460    0.6965
    0.3960    0.4000    0.6711    0.5920
         0    0.4168    0.3502    0.2955
    0.4168         0    0.4000    0.4590
    0.3502    0.4000         0    0.3921
    0.2955    0.4590    0.3921         0


vaf =

    0.9906


targone =

  Columns 1 through 6

         0    0.3148    0.6038    0.6296    0.6296    0.7457
    0.3148         0    0.1778    0.5201    0.5201    0.6626
    0.6038    0.1778         0    0.2742    0.3230    0.5028
    0.6296    0.5201    0.2742         0    0.3192    0.5012
    0.6296    0.5201    0.3230    0.3192         0    0.2831
    0.7457    0.6626    0.5028    0.5012    0.2831         0
    0.7457    0.6626    0.5028    0.5012    0.3340    0.3021
    0.7936    0.7061    0.6997    0.6974    0.6027    0.5526
    0.7936    0.7061    0.6997    0.6974    0.6027    0.5526
    0.7936    0.7061    0.6997    0.6974    0.6027    0.5527

  Columns 7 through 10

    0.7457    0.7936    0.7936    0.7936
    0.6626    0.7061    0.7061    0.7061
    0.5028    0.6997    0.6997    0.6997
    0.5012    0.6974    0.6974    0.6974
    0.3340    0.6027    0.6027    0.6027
    0.3021    0.5526    0.5526    0.5527
         0    0.3229    0.3229    0.4963
    0.3229         0    0.2815    0.4197
    0.3229    0.2815         0    0.3001
    0.4963    0.4197    0.3001         0


targtwo =

  Columns 1 through 6

         0   -0.3567   -0.2640    0.0542    0.0542    0.0938
   -0.3567         0   -0.3327    0.0272    0.0272    0.0919
   -0.2640   -0.3327         0   -0.0198   -0.0198    0.0799
    0.0542    0.0272   -0.0198         0   -0.0198    0.0799
    0.0542    0.0272   -0.0198   -0.0198         0   -0.2008
    0.0938    0.0919    0.0799    0.0799   -0.2008         0
    0.0938    0.0919    0.0799    0.0799   -0.2008   -0.4344
    0.1260    0.1185    0.1062    0.1062    0.0939   -0.0811
    0.1260    0.1185    0.1062    0.1062    0.0939    0.0393
    0.1260    0.1185    0.1062    0.1062    0.0939    0.0393

  Columns 7 through 10

    0.0938    0.1260    0.1260    0.1260
    0.0919    0.1185    0.1185    0.1185
    0.0799    0.1062    0.1062    0.1062
    0.0799    0.1062    0.1062    0.1062
   -0.2008    0.0939    0.0939    0.0939
   -0.4344   -0.0811    0.0393    0.0393
         0   -0.1741   -0.0907   -0.0734
   -0.1741         0   -0.0907   -0.0734
   -0.0907   -0.0907         0   -0.1526
   -0.0734   -0.0734   -0.1526         0


outpermone =

     1     2     3     4     5     6     7     8     9    10


outpermtwo =

     5     9     3     1     7    10     4     2     8     6
\end{verbatim}

\large

\section[Circular-Anti-Robinson (CAR) Matrices]{Circular-Anti-Robinson  (CAR) Matrices for Symmetric Proximity Data}


In the approximation of a proximity matrix $\mathbf{P}$ by one
that is row/column reorderable to an AR form, the interpretation
of the fitted matrix in general had to be carried out  by
identifying a set of subsets through an increasing threshold
variable; each of the subsets contained objects that were
contiguous with respect to a given \emph{linear}  ordering along
a
continuum, and had a diameter defined by the maximum fitted value
within the subset.  To provide a further representation depicting the fitted values as lengths of paths in a graph, an
approximation was sought that satisfied the additional
constraints of an SAR matrix; still, the subsets thus identified
had
to contain objects contiguous with respect to a linear
ordering.  As one possible generalization of both
the AR and SAR constraints, we can define what will be called
circular anti-Robinson (CAR) and circular strongly-anti-Robinson
(CSAR) forms that allow the subsets identified from
increasing a threshold variable to be contiguous with respect to
a
\emph{circular} ordering of the objects around a closed
continuum.  Approximation matrices that are row/column
reorderable to display an AR or SAR form, respectively, will also
be (trivially) row/column reorderable to display what is
formally characterized below as a CAR  or a CSAR form, but not
conversely.  (Historically, there is a large literature on the possibility of circular structures emerging from and being identifiable in a given proximity matrix.  For a variety of references,  the reader is referred to the American Psychological Association sponsored volume edited by Plutchik and Conte (1997), or the discussion of metric circular unidimensional scaling from the Unidimensional Scaling Toolbox.   The extension of CAR forms to those that are also CSAR, however, has apparently not been a topic discussed in the literature before the appearance of Hubert, Arabie, and Meulman [1998];    this latter source forms the basis for much of the present section.)


\smallskip


To be explicit, an arbitrary symmetric matrix $\mathbf{Q} =
\{q_{ij}\}$, where $q_{ii} = 0$ for $1 \leq i,j \leq n$, is said
to be row/column reorderable to a circular anti-Robinson form
(or, for short, $\mathbf{Q}$ is a circular anti-Robinson (CAR)
matrix) if there exists a permutation, $\rho(\cdot)$, on the
first $n$ integers such that the reordered matrix
$\mathbf{Q}_{\rho} = \{q_{\rho(i) \rho(j) }\}$ satisfies the
conditions given in (II):

\medskip

\noindent (II):  for $1 \leq i \leq n-3$, and $i+1 < j \leq n-1$,

\smallskip

if $q_{\rho(i+1) \rho(j)} \leq  q_{\rho(i) \rho(j+1)}$, then

$q_{\rho(i+1) \rho(j)} \leq  q_{\rho(i) \rho(j)}$ and
$q_{\rho(i+1) \rho(j)} \leq  q_{\rho(i+1) \rho(j+1)}$;

\smallskip

if $q_{\rho(i+1) \rho(j)} \geq  q_{\rho(i) \rho(j+1)}$, then

$q_{\rho(i) \rho(j)} \geq  q_{\rho(i) \rho(j+1)}$ and
$q_{\rho(i+1) \rho(j+1)} \geq  q_{\rho(i)
 \rho(j+1)}$,

\smallskip


and, for $2 \leq i \leq n-2$,

\smallskip

if $q_{\rho(i+1) \rho(n)} \leq  q_{\rho(i) \rho(1)}$, then

$q_{\rho(i+1) \rho(n)} \leq  q_{\rho(i) \rho(n)}$ and
$q_{\rho(i+1) \rho(n)} \leq  q_{\rho(i+1) \rho(1)}$;

\smallskip

if $q_{\rho(i+1) \rho(n)} \geq  q_{\rho(i) \rho(1)}$, then

$q_{\rho(i) \rho(n)} \geq  q_{\rho(i) \rho(1)}$ and
$q_{\rho(i+1) \rho(1)} \geq  q_{\rho(i) \rho(1)}$.

\medskip

\noindent Interpretatively, within each row of
$\mathbf{Q}_{\rho}$ moving to the right from the main diagonal
and then wrapping back around to re-enter the same row from the
left, the entries never
decrease until a maximum is reached and then never increase
moving away from the maximum until the main diagonal is again
reached.  Given the symmetry of $\mathbf{P}$, a similar pattern
of entries
would be present within each column as well.  As noted above, any
AR matrix is CAR but not conversely.

\smallskip

In analogy to the SAR conditions that permit graphical
representation, a symmetric matrix $\mathbf{Q}$ is said to be
row/column reorderable to a circular strongly-anti-Robinson form
(or, for short, $\mathbf{Q}$ is a \emph{circular strongly-anti-Robinson}
(CSAR) matrix) if there exists a permutation, $\rho(\cdot)$, on
the first $n$ integers such that the reordered matrix
$\mathbf{Q}_{\rho} = \{q_{\rho(i) \rho(j)} \}$ satisfies the
conditions given by (II), \emph{and}

\medskip

for $1 \leq i \leq n-3$, and $i+1 < j \leq n-1$,

\smallskip

if $q_{\rho(i+1) \rho(j)} \leq  q_{\rho(i) \rho(j+1)}$,

 then
$q_{\rho(i+1) \rho(j)} =  q_{\rho(i) \rho(j)}$ im\-plies
$q_{\rho(i+1) \rho(j+1)} =  q_{\rho(i) \rho(j+1)}$, and
$q_{\rho(i+1) \rho(j)} =  q_{\rho(i+1) \rho(j+1)}$ im\-plies
 $q_{\rho(i) \rho(j)} =  q_{\rho(i) \rho(j+1)}$;

\smallskip

if $q_{\rho(i+1) \rho(j)} \geq  q_{\rho(i) \rho(j+1)}$,

then
$q_{\rho(i) \rho(j+1)} =  q_{\rho(i+1) \rho(j+1)}$ im\-plies
$q_{\rho(i) \rho(j)} =  q_{\rho(i+1) \rho(j)}$, and
$q_{\rho(i) \rho(j)} =  q_{\rho(i) \rho(j+1)}$ im\-plies
$q_{\rho(i+1) \rho(j)} =  q_{\rho(i+1) \rho(j+1)}$,

\medskip

and for $2 \leq i \leq n - 2$,

\smallskip


if $q_{\rho(i+1) \rho(n)} \leq  q_{\rho(i) \rho(1)}$, then
$q_{\rho(i+1) \rho(n)} =  q_{\rho(i) \rho(n)}$ im\-plies
$q_{\rho(i+1) \rho(1)} =  q_{\rho(i) \rho(1)}$, and
$q_{\rho(i+1) \rho(n)}$ $= q_{\rho(i+1) \rho(1)}$ im\-plies
$q_{\rho(i) \rho(n)} =  q_{\rho(i) \rho(1)}$;

\smallskip

if $q_{\rho(i+1) \rho(n)} \geq  q_{\rho(i) \rho(1)}$, then
$q_{\rho(i) \rho(1)}$ = $q_{\rho(i+1) \rho(1)}$ i\-mplies
$q_{\rho(i) \rho(n)}$ = $q_{\rho(i+1) \rho(n)}$, and
$q_{\rho(i) \rho(n)}$ = $q_{\rho(i) \rho(1)}$ im\-plies
$q_{\rho(i+1) \rho(n)}$ = $q_{\rho(i+1) \rho(1)}$.


\medskip

\noindent Again, the imposition of the stronger CSAR conditions
avoids the type
of graphical anomaly present in Figure 1(b) but now in the
context of a CAR
matrix --- when two fitted values that are adjacent within a row
are equal, the
fitted values in the same two adjacent columns must also be equal
for a row that
is either its immediate predecessor (if $q_{\rho(i+1) \rho(j)}
\leq q_{\rho(i)
\rho(j+1)}$), or successor (if $q_{\rho(i+1) \rho(j)} \geq
q_{\rho(i)
\rho(j+1)}$); a similar condition is imposed when two fitted
values that are
adjacent within a column are equal.  As noted,
any SAR matrix is CSAR but not conversely.

\smallskip


The computational strategy we suggest for identifying a
best-fitting CAR or CSAR approximation matrix is based on an
initial
circular unidimensional scaling obtained through the optimization
strategy developed by Hubert, Arabie, and Meulman (1997).
Specifically, we first institute a combination of combinatorial search for good
matrix reorderings and heuristic iterative projection to locate
the points of inflection when  minimum distance calculations
change directionality around a closed circular structure.
Approximation matrices to $\mathbf{P}$ are found through a
least-squares loss criterion, and they have the parameterized form
\[\mathbf{Q}_{\rho} = \{ \min(\mid x_{\rho(j)} - x_{\rho(i)} \mid
, \  x_{0} - \mid x_{\rho(j)} - x_{\rho(i)} \mid ) \ +c\} ,\]
where $c$ is an estimated additive constant, $x_{\rho(1)} \leq
x_{\rho(2)} \leq \cdots \leq x_{\rho(n)} \leq x_{0}$, and the
last
coordinate, $x_{0}$, is the circumference of the circular
structure.  Based on the inequality constraints implied by such a
collection of coordinates, a CAR approximation matrix can be
fitted
to $\mathbf{P}$ directly; then, beginning with this latter CAR
approximation, the identification and imposition of CSAR
constraints  proceeds through the heuristic use of iterative
projection, directly analogous to the way SAR constraints
in the linear  ordering context were identified and fitted,
beginning with a best approximation matrix satisfying just the
AR restrictions.

\subsection{Fitting a Given CAR Matrix in the $L_{2}$-Norm}


The  function M-file, \verb+cirarobfit.m+, fits a circular anti-Robinson (CAR) matrix using iterative projection to
 a symmetric proximity matrix in the $L_{2}$-norm.  Usage syntax is
\begin{verbatim}
[fit, vaf] = cirarobfit(prox,inperm,targ)
\end{verbatim}
\noindent where \verb+PROX+ is the input proximity matrix ($n \times n$ with a zero main diagonal
 and a dissimilarity interpretation);
 \verb+INPERM+ is a given permutation of the first $n$ integers (around a circle);
 \verb+TARG+ is a given $n \times n$ matrix having the circular anti-Robinson
 form that guides the direction in which distances are taken around the circle.
 The matrix \verb+FIT+ is the least-squares optimal approximation (with variance-accounted-for
 of \verb+VAF+) to \verb+PROX+ having an circular anti-Robinson form for the row and column
 object ordering given by \verb+INPERM+.

 \smallskip

A recording of a MATLAB  session follows that uses the \verb+number.dat+ data file; an equally-spaced circular anti-Robinson matrix \verb+targcircular+ obtained from the utility M-file, \verb+targcir.m+; and the identity permutation for the objects around the circular structure.  The fitted CAR matrix thus identified in this way has a \verb+VAF+ of 64.37\%.

\tiny

\begin{verbatim}
 load number.dat
 targcircular = targcir(10);
 [fit vaf] = cirarobfit(number,1:10,targcircular)

fit =

  Columns 1 through 6

         0    0.4210    0.5840    0.6510    0.6835    0.8040
    0.4210         0    0.2840    0.3460    0.6170    0.6170
    0.5840    0.2840         0    0.2753    0.2753    0.5460
    0.6510    0.3460    0.2753         0    0.2753    0.3844
    0.6835    0.6170    0.2753    0.2753         0    0.3844
    0.8040    0.6170    0.5460    0.3844    0.3844         0
    0.7730    0.7730    0.5460    0.3844    0.3844    0.3844
    0.7695    0.7695    0.7960    0.5920    0.5530    0.4000
    0.6597    0.6597    0.6597    0.8040    0.5530    0.5530
    0.6510    0.6510    0.6510    0.6510    0.6835    0.5920

  Columns 7 through 10

    0.7730    0.7695    0.6597    0.6510
    0.7730    0.7695    0.6597    0.6510
    0.5460    0.7960    0.6597    0.6510
    0.3844    0.5920    0.8040    0.6510
    0.3844    0.5530    0.5530    0.6835
    0.3844    0.4000    0.5530    0.5920
         0    0.3857    0.3857    0.3857
    0.3857         0    0.3857    0.3857
    0.3857    0.3857         0    0.3857
    0.3857    0.3857    0.3857         0


vaf =

    0.6437
\end{verbatim}

\large


\subsection{Finding a CAR Matrix in the $L_{2}$-Norm}

The M-file, \verb+cirarobfnd.m+, is our suggested strategy for identifying a best-fitting CAR matrix for
a symmetric proximity matrix in the $L_{2}$-norm based on a permutation that is
initially identified through the use of iterative quadratic assignment.  Based on an equally-spaced circular target matrix, \verb+order.m+ is first invoked to obtain a good (circular) permutation, which in turn is then used to construct a new circular target matrix with \verb+cirfit.m+. (We will mention here but not illustrate with an example, an alternative to the use of \verb+cirarobfnd.m+ called \verb+cirarobfnd_ac.m+; the latter M-file has the same syntax as
\verb+cirarobfnd.m+ but uses \verb+cirfitac.m+ rather than \verb+cirfit.m+ internally to obtain the new circular target matrices.)   The final output is generated from \verb+cirarobfit.m+  that no better permutation can be identified using the newer circular target matrix.   The usage syntax for \verb+cirarobfnd.m+ is as follows:
\begin{verbatim}
[find, vaf, outperm] = cirarobfnd(prox, inperm, kblock)
\end{verbatim}
\noindent where
 \verb+PROX+ is the input proximity matrix ($n \times n$ with a zero main diagonal
 and a dissimilarity interpretation);
 \verb+INPERM+ is a given starting permutation (assumed to be around the
 circle) of the first $n$ integers;
 \verb+FIND+ is the least-squares optimal matrix (with variance-accounted-for
 of \verb+VAF+) to \verb+PROX+ having a circular anti-Robinson form for the row and column
 object ordering given by the concluding permutation \verb+OUTPERM+. Again, \verb+KBLOCK+
 defines the block size in the use of the iterative quadratic assignment
 routine.

 \smallskip

An example of the use of \verb+cirarobfnd.m+ is given below that seems to lead to a circular ordering best interpreted according to the structural properties of the digits.  This solution is only one of several local optima identifiable by repeated application of  the routine using other random starting permutations. In general, the different local
optima observed differ in the way the odd digits,
$\{3,5,7,9\}$, and the even digits, $\{2,4,6,8\}$, are ordered
within these sets when moving clockwise around a circular
structure.
Explicitly, all local optima had a general structure of
$\rightarrow 0 \rightarrow 1 \rightarrow \{3,5,7,9\} \rightarrow
\{2,4,6,8\} \rightarrow$, but with some variation in order within
the odd and even digits.  For example, the CAR matrix given below
uses the odd digits as $\rightarrow 3 \rightarrow 5 \rightarrow
9 \rightarrow 7 \rightarrow$ and the even digits as
$\rightarrow 6 \rightarrow 8 \rightarrow 4 \rightarrow 2
\rightarrow$.

\tiny

\begin{verbatim}
[find, vaf, outperm] = cirarobfnd(number, randperm(10), 3)

find =

  Columns 1 through 6

         0    0.3460    0.5315    0.5315    0.6069    0.8040
    0.3460         0    0.4210    0.4340    0.6069    0.7895
    0.5315    0.4210         0    0.4340    0.6069    0.7895
    0.5315    0.4340    0.4340         0    0.0590    0.3670
    0.6069    0.6069    0.6069    0.0590         0    0.2460
    0.8040    0.7895    0.7895    0.3670    0.2460         0
    0.4460    0.7895    0.7895    0.4210    0.3880    0.3500
    0.4460    0.6300    0.9090    0.7697    0.6069    0.3960
    0.4160    0.6250    0.8500    0.7698    0.6069    0.3960
    0.4160    0.5880    0.7698    0.7698    0.6069    0.6069

  Columns 7 through 10

    0.4460    0.4460    0.4160    0.4160
    0.7895    0.6300    0.6250    0.5880
    0.7895    0.9090    0.8500    0.7698
    0.4210    0.7697    0.7698    0.7698
    0.3880    0.6069    0.6069    0.6069
    0.3500    0.3960    0.3960    0.6069
         0    0.3907    0.3907    0.4160
    0.3907         0    0.3907    0.4160
    0.3907    0.3907         0    0.4160
    0.4160    0.4160    0.4160         0


vaf =

    0.8128


outperm =

     4     2     1     3     5     9     7     8    10     6
\end{verbatim}

\large

\subsection{Finding a Circular Strongly-Anti-Robinson (CSAR) Matrix in the $L_{2}$-Norm}

The two M-functions, \verb+cirsarobfit.m+ and \verb+cirsarobfnd.m+, are direct analogues of \verb+cirarobfit.m+ and \verb+cirarobfnd.m+, respectively, but are concerned with fitting and finding  \emph{strongly} circular-anti-Robinson forms (also, we mention but do not illustrate, the M-file \verb+cirsarobfnd_ac.m+ which uses \verb+cirarobfnd_ac.m+ to obtain the initial CAR matrix that is then strengthened into one that is CSAR).  The syntax for \verb+cirsarobfit.m+, which fits a circular strongly-anti-Robinson matrix using iterative projection to
 a symmetric proximity matrix in the $L_{2}$-norm, is
\begin{verbatim}
[fit, vaf] = cirsarobfit(prox, inperm, targ)
\end{verbatim}
\noindent where, again, \verb+PROX+ is the input proximity matrix ($n \times n$ with a zero main diagonal
 and a dissimilarity interpretation); \verb+INPERM+ is a given permutation of the first $n$ integers; \verb+TARG+ is a given $n \times n$ matrix having the circular anti-Robinson
 form that guides the direction in which distances are taken around the circle.
 \verb+FIT+ is the least-squares optimal matrix (with variance-accounted-for
 of \verb+VAF+) to \verb+PROX+ having a strongly circular-anti-Robinson form for the row and column
 object ordering given by \verb+INPERM+.

 \smallskip

An example follows using the same identity permutation as in fitting a CAR form with \verb+cirarobfit.m+; as might be expected from using the more restrictive CSAR form, the variance-accounted-for drops to .4501 from .6437.

\tiny

\begin{verbatim}
[fit, vaf] = cirsarobfit(number,1:10,targcircular)

fit =

  Columns 1 through 6

         0    0.4210    0.5840    0.6505    0.6505    0.6505
    0.4210         0    0.2840    0.6505    0.6505    0.6505
    0.5840    0.2840         0    0.2753    0.2753    0.4306
    0.6505    0.6505    0.2753         0    0.2753    0.4306
    0.6505    0.6505    0.2753    0.2753         0    0.4306
    0.6505    0.6505    0.4306    0.4306    0.4306         0
    0.6505    0.6505    0.4306    0.4306    0.4306    0.4306
    0.6505    0.6505    0.6505    0.6505    0.6505    0.6505
    0.6505    0.6505    0.6505    0.6505    0.6505    0.6505
    0.6505    0.6505    0.6505    0.6505    0.6505    0.6505

  Columns 7 through 10

    0.6505    0.6505    0.6505    0.6505
    0.6505    0.6505    0.6505    0.6505
    0.4306    0.6505    0.6505    0.6505
    0.4306    0.6505    0.6505    0.6505
    0.4306    0.6505    0.6505    0.6505
    0.4306    0.6505    0.6505    0.6505
         0    0.3857    0.3857    0.3857
    0.3857         0    0.3857    0.3857
    0.3857    0.3857         0    0.3857
    0.3857    0.3857    0.3857         0


vaf =

    0.4501
\end{verbatim}

\large

The M-function, \verb+cirsarobfnd.m+, finds and fits a CSAR matrix using iterative projection to
 a symmetric proximity matrix in the $L_{2}$-norm based on a permutation
 identified through the use of iterative quadratic assignment.  It has the expected syntax
\begin{verbatim}
[find, vaf, outperm] = cirsarobfnd(prox, inperm, kblock)
\end{verbatim}
 \noindent where, again, \verb+PROX+ is the input proximity matrix ($n \times n$ with a zero main diagonal
 and a dissimilarity interpretation); \verb+INPERM+ is a given starting permutation of the first $n$ integers;
 \verb+FIND+ is the least-squares optimal matrix (with variance-accounted-for
 of \verb+VAF+) to \verb+PROX+ having a circular strongly-anti-Robinson form for the row and column
 object ordering given by the ending permutation \verb+OUTPERM+. As usual, \verb+KBLOCK+
 defines the block size in the use the iterative quadratic assignment
 routine. (Analogous to the last section, and as noted above, an alternative to \verb+cirsarobfnd.m+
is available, called \verb+cirsarobfnd_ac.m+  that uses \verb+cirfitac.m+ to obtain the circular target matrices.)

\smallskip

In the MATLAB  recording below, and starting from a random permutation, a circular strongly anti-Robinson form was found with a \verb+VAF+ of .7296 (again, this represents an expected drop from the value of .8119 for the CAR form --- this result is also listed below)).

\tiny


\begin{verbatim}
[find, vaf, outperm] = cirsarobfnd(number,randperm(10), 2)

target =

  Columns 1 through 6

         0    0.4160    0.4160    0.4160    0.6262    0.6262
    0.4160         0    0.3907    0.3907    0.3960    0.6263
    0.4160    0.3907         0    0.3907    0.3960    0.6263
    0.4160    0.3907    0.3907         0    0.3500    0.3880
    0.6262    0.3960    0.3960    0.3500         0    0.2460
    0.6262    0.6263    0.6263    0.3880    0.2460         0
    0.7858    0.7858    0.7858    0.4210    0.3670    0.0590
    0.7858    0.7858    0.9090    0.7895    0.7895    0.5810
    0.5880    0.6250    0.6300    0.7895    0.7895    0.5810
    0.4160    0.4160    0.4460    0.4460    0.8040    0.5810

  Columns 7 through 10

    0.7858    0.7858    0.5880    0.4160
    0.7858    0.7858    0.6250    0.4160
    0.7858    0.9090    0.6300    0.4460
    0.4210    0.7895    0.7895    0.4460
    0.3670    0.7895    0.7895    0.8040
    0.0590    0.5810    0.5810    0.5810
         0    0.4340    0.4340    0.5315
    0.4340         0    0.4210    0.5315
    0.4340    0.4210         0    0.3460
    0.5315    0.5315    0.3460         0


vaf =

    0.8119


outperm =

     6    10     8     7     9     5     3     1     2     4


find =

  Columns 1 through 6

         0    0.4246    0.4246    0.4246    0.7304    0.7304
    0.4246         0    0.3907    0.3907    0.3960    0.7304
    0.4246    0.3907         0    0.3907    0.3960    0.7304
    0.4246    0.3907    0.3907         0    0.3500    0.3880
    0.7304    0.3960    0.3960    0.3500         0    0.2460
    0.7304    0.7304    0.7304    0.3880    0.2460         0
    0.7304    0.7304    0.7304    0.4210    0.3670    0.0590
    0.7304    0.7304    0.7304    0.7304    0.7304    0.5810
    0.7304    0.7304    0.7304    0.7304    0.7304    0.5810
    0.4246    0.4246    0.4246    0.4246    0.7304    0.5810

  Columns 7 through 10

    0.7304    0.7304    0.7304    0.4246
    0.7304    0.7304    0.7304    0.4246
    0.7304    0.7304    0.7304    0.4246
    0.4210    0.7304    0.7304    0.4246
    0.3670    0.7304    0.7304    0.7304
    0.0590    0.5810    0.5810    0.5810
         0    0.4340    0.4340    0.5315
    0.4340         0    0.4210    0.5315
    0.4340    0.4210         0    0.3460
    0.5315    0.5315    0.3460         0


vaf =

    0.7296


outperm =

     6    10     8     7     9     5     3     1     2     4
\end{verbatim}

\large

\subsection{Graphically Representing CSAR Structures}


As in the case of an AR or SAR matrix, the interpretation of the
structure that may be represented by a CAR or CSAR matrix could
proceed by first identifying those subsets and their diameters
that emerge by increasing a threshold variable from the smallest fitted value.  And in
the case of a more restrictive CSAR matrix, this collection of
subsets and their diameters can then be displayed by a  graph
where minimum length paths reconstruct the fitted values.  To
illustrate this graphical possibility on the transformed \verb+number.dat+ to mean 4.0 and variance 1.0 given in Hubert, Arabie, and Meulman (1998)  --- and used earlier to show the graphical representation of an SAR matrix ---
 the fifteen (nonredundant) subsets identified from the
CSAR matrix present in Table 7 are listed in Table 6 according to increasing diameter.
 Here, the structural properties of the digits are
apparent (e.g., various subsets of the odd or even digits, or
those
that are multiples or powers of 2 or of 3), but some magnitude
adjacencies can also be noted (e.g., $\{6,7,8,9\}$, or subsets of
$\{0,1,2,3\}$).  The graph adhering to the
CSAR restrictions is given in Figure 3 and again
minimum path lengths (that proceed up from a terminal node to an
internal node and then back down to the other terminal node) can
be used to reconstruct the fitted values
in $\mathbf{Q}$.

\smallskip

In addition to searching for a best-fitting CSAR matrix
directly, we might comment that the type of indirect
approach mentioned in the introduction for the case of SAR approximations could also
be considered, although we will not go into the details
here.  For example, based on a best-fitting CAR matrix, the
additional constraints of a circular unidimensional scale could
be identified and then imposed (in fact, this is our starting
place in first obtaining the CAR approximation); or
those of an
ultrametric
(which would lead to an SAR matrix that is trivially CSAR as
well); or possibly, a collection of additive tree restrictions
could be identified.  In all cases, CSAR approximations would be
automatically obtained.

\smallskip

\begin{table}


\caption{The fifteen (nonredundant) subsets listed according to increasing diameter values  are contiguous in the circular object ordering used to display the CSAR entries in Table 7.}

\bigskip
\begin{center}

\begin{tabular}{lcclc}

\emph{subset} & \emph{diameter} & \hspace{3ex} & \emph{subset} &
\emph{diameter}
\\
\{4,2\} & 1.63 & \hspace{3ex} & \{6,8,4,2\} & 3.41 \\
\{8,4\} & 2.55 & \hspace{3ex} & \{0,1\} & 3.41 \\
\{1,3\} & 3.04 & \hspace{3ex} & \{3,5,9,7,6\} & 3.43 \\
\{6,8\} & 3.06 & \hspace{3ex} & \{2,0,1\} & 3.47 \\
\{8,4,2\} & 3.14 & \hspace{3ex} & \{2,0,1,3\} & 3.95 \\
\{6,8,4\} & 3.25 & \hspace{3ex} & \{4,2,0,1,3\} & 4.20 \\
\{9,7,6\} & 3.26 &  \hspace{3ex} & \{0,1,3,5,9,7,6,8,4,2\}  &
4.93 \\
\{9,7,6,8\} & 3.29 & \hspace{3ex} & & \\

\end{tabular}

\end{center}

\end{table}


\begin{table}




\caption{A circular strongly-anti-Robinson order-constrained least-squares approximation to the
digit proximity data of Shepard \emph{et al}.\ (1975).}

\bigskip

\begin{center}

\begin{tabular}{lcccccccccc}

digit & 0 & 1 & 3 & 5 & 9 & 7 & 6 & 8 & 4 & 2  \\
0 &    x & 3.41 & 3.95 & 4.93 & 4.93 & 4.93 & 4.93 & 4.93 & 4.20 & 3.47  \\
1 & 3.41 &    x & 3.04 & 4.93 & 4.93 & 4.93 & 4.93 & 4.93 & 4.20 & 3.47  \\
3 & 3.95 & 3.04 &    x & 3.43 & 3.43 & 3.43 & 3.43 & 4.93 & 4.20 & 3.95  \\
5 & 4.93 & 4.93 & 3.43 &    x & 3.43 & 3.43 & 3.43 & 4.93 & 4.93 & 4.93  \\
9 & 4.93 & 4.93 & 3.43 & 3.43 &    x & 3.26 & 3.26 & 3.29 & 4.93 & 4.93  \\
7 & 4.93 & 4.93 & 3.43 & 3.43 & 3.26 &    x & 3.26 & 3.29 & 4.93 & 4.93  \\
6 & 4.93 & 4.93 & 3.43 & 3.43 & 3.26 & 3.26 &    x & 3.06 & 3.25 & 3.41  \\
8 & 4.93 & 4.93 & 4.93 & 4.93 & 3.29 & 3.29 & 3.06 &    x & 2.55 & 3.14  \\
4 & 4.20 & 4.20 & 4.20 & 4.93 & 4.93 & 4.93 & 3.25 & 2.55 &    x & 1.63  \\
2 & 3.47 & 3.47 & 3.95 & 4.93 & 4.93 & 4.93 & 3.41 & 3.14 & 1.63 & x  \\

\end{tabular}


\end{center}

\end{table}

\setlength{\unitlength}{1.00mm}

\begin{figure}

\begin{center}


\begin{picture}(125,65)(-10,-10)





\put(10,0){\circle{2}}
 \put(20,0){\circle{2}}
\put(30,0){\circle{2}}
 \put(40,0){\circle{2}}
\put(50,0){\circle{2}}
 \put(60,0){\circle{2}}
\put(70,0){\circle{2}}
 \put(80,0){\circle{2}}
\put(90,0){\circle{2}}
\put(100,0){\circle{2}}
\put(110,0){\circle{2}}
\put(20,34.3){\circle*{2}}
 \put(40,32.6){\circle*{2}}
\put(47.5,32.9){\circle*{2}}
 \put(60,32.5){\circle*{2}}
\put(55,30.6){\circle*{2}}
\put(65,25.5){\circle*{2}}
\put(70,31.4){\circle*{2}}
 \put(75,16.3){\circle*{2}}
\put(65,34.1){\circle*{2}}
 \put(87.5,34.7){\circle*{2}}
\put(95,34.1){\circle*{2}}
 \put(105,30.4){\circle*{2}}
\put(96.3,39.5){\circle*{2}}
 \put(85.6,42.0){\circle*{2}}
\put(52.8,49.3){\circle*{2}}

\put(11,0){\line(0,1){34.3}}
\put(20,1){\line(0,1){33.3}}
\put(31,0){\line(0,1){32.6}}
\put(40,1){\line(0,1){31.6}}
\put(49,0){\line(0,1){32.6}}
 \put(51,0){\line(0,1){30.6}}
\put(59,0){\line(0,1){30.6}}
\put(61,0){\line(0,1){25.5}}
\put(69,0){\line(0,1){25.5}}
 \put(71,0){\line(0,1){16.3}}
\put(79,0){\line(0,1){16.3}}
\put(81,0){\line(0,1){34.7}}
\put(91,0){\line(0,1){34.1}}
 \put(99,0){\line(0,1){34.1}}
\put(101,0){\line(0,1){30.4}}
 \put(109,0){\line(0,1){30.4}}
\put(104,30.4){\line(0,1){9.1}}
 \put(94,34.1){\line(0,1){.6}}
\put(88.5,34.7){\line(0,1){4.8}}
\put(95.3,39.5){\line(0,1){2.5}}
\put(84.6,42.0){\line(0,1){7.3}}
 \put(56,30.6){\line(0,1){1.9}}
\put(64,25.5){\line(0,1){7.0}}
 \put(69,31.4){\line(0,1){2.7}}
\put(74,16.3){\line(0,1){15.1}}
\put(76,16.3){\line(0,1){25.7}}
\put(66,25.5){\line(0,1){5.9}}
\put(54,30.6){\line(0,1){2.3}}
 \put(64,34.1){\line(0,1){15.2}}
\put(46.5,32.9){\line(0,1){16.4}}
\put(39,32.6){\line(0,1){1.7}}
\put(21,34.3){\line(0,1){15.0}}
\put(61,32.5){\line(0,1){1.6}}



\put(11,34.3){\line(1,0){28.0}}
 \put(31,32.6){\line(1,0){18.0}}
\put(41,32.9){\line(1,0){8.0}}
\put(51,30.6){\line(1,0){8.0}}
\put(61,25.5){\line(1,0){8.0}}
 \put(71,16.3){\line(1,0){8.0}}
\put(101,30.4){\line(1,0){8.0}}
 \put(91,34.1){\line(1,0){8.0}}
\put(81,34.7){\line(1,0){13.0}}
 \put(88.5,39.5){\line(1,0){15.5}}
\put(76,42.0){\line(1,0){19.3}}
 \put(21,49.3){\line(1,0){63.6}}
\put(61,34.1){\line(1,0){8.0}}
 \put(66,31.4){\line(1,0){8.0}}
\put(56,32.5){\line(1,0){8.0}}
\put(41,32.9){\line(1,0){13.0}}




\put(10,-2){\makebox(0,0)[t]{3}}
\put(20,-2){\makebox(0,0)[t]{5}}
\put(30,-2){\makebox(0,0)[t]{9}}
\put(40,-2){\makebox(0,0)[t]{7}}
\put(50,-2){\makebox(0,0)[t]{6}}
\put(60,-2){\makebox(0,0)[t]{8}}
\put(70,-2){\makebox(0,0)[t]{4}}
\put(80,-2){\makebox(0,0)[t]{2}}
\put(90,-2){\makebox(0,0)[t]{0}}
\put(100,-2){\makebox(0,0)[t]{1}}
\put(110,-2){\makebox(0,0)[t]{3}}

\curvedashes[1pt]{0,1,2}
\put(10,-6){\curve(1,0, 50,-10, 100,0)}
\curvedashes{}

\put(0.0,16.3){\line(1,0){4.0}}
 \put(0.0,25.5){\line(1,0){4.0}}
\put(0.0,30.4){\line(1,0){4.0}}
\put(0.0,30.6){\line(1,0){4.0}}
\put(0.0,31.4){\line(1,0){4.0}}
 \put(0.0,32.5){\line(1,0){4.0}}
\put(0.0,32.6){\line(1,0){4.0}}
 \put(0.0,32.9){\line(1,0){4.0}}
\put(0.0,34.1){\line(1,0){4.0}}
 \put(0.0,34.3){\line(1,0){4.0}}
\put(0.0,34.7){\line(1,0){4.0}}
 \put(0.0,39.5){\line(1,0){4.0}}
\put(0.0,42.0){\line(1,0){4.0}}
 \put(0.0,49.3){\line(1,0){4.0}}


\put(-4.0,15.0){\makebox(0,0)[br]{1.5}}
\put(-4.0,20.0){\makebox(0,0)[br]{2.0}}
\put(-4.0,25.0){\makebox(0,0)[br]{2.5}}
\put(-4.0,30.0){\makebox(0,0)[br]{3.0}}
\put(-4.0,35.0){\makebox(0,0)[br]{3.5}}
\put(-4.0,40.0){\makebox(0,0)[br]{4.0}}
\put(-4.0,45.0){\makebox(0,0)[br]{4.5}}
\put(-4.0,50.0){\makebox(0,0)[br]{5.0}}

\put(-4.0,15.0){\vector(1,0){3}}
 \put(-4.0,20.0){\vector(1,0){3}}
\put(-4.0,25.0){\vector(1,0){3}}
 \put(-4.0,30.0){\vector(1,0){3}}
\put(-4.0,35.0){\vector(1,0){3}}
 \put(-4.0,40.0){\vector(1,0){3}}
\put(-4.0,45.0){\vector(1,0){3}}
 \put(-4.0,50.0){\vector(1,0){3}}

\put(0.0,0.0){\line(0,1){50.0}}

\end{picture}

\bigskip

\caption{A graphical representation for the fitted values given
by the circular strongly-anti-Robinson matrix in the
lower-triangular
portion of Table 7 (VAF = 72.96\%).  Note that digit 3 is
placed both in the
first and the last positions in the ordering of the objects with
the implication that the sequence  continues in a circular
manner.  This circularity is indicated by the curved dashed line.}

\end{center}


\end{figure}




\subsection{Representation Through Multiple (Strongly) CAR Matrices}

Just as we discussed  representing of proximity matrices through multiple (strongly) AR matrices, the analysis of a proximity matrix by a single (strongly) circular-anti-Robinson structure extends easily to the additive use of multiple matrices.  The M-function, \verb+bicirarobfnd.m+, fits the sum of two circular-anti-Robinson matrices using iterative projection to a symmetric proximity matrix in the $L_{2}$-norm based on permutations
 identified through the use of iterative quadratic assignment.  The syntax usage is
\begin{verbatim}
[find,vaf,targone,targtwo,outpermone,outpermtwo] = ...
bicirarobfnd(prox,inperm,kblock)
\end{verbatim}
 \noindent where, as before, \verb+PROX+ is the input proximity matrix ($n \times n$ with a zero main diagonal
 and a dissimilarity interpretation);
 \verb+INPERM+ is a given initial permutation of the first $n$ integers;
 \verb+FIND+ is the least-squares optimal matrix (with variance-accounted-for
 of \verb+VAF+) to \verb+PROX+ and is the sum of the two circular-anti-Robinson matrices
 \verb+TARGONE+ and \verb+TARGTWO+ based on the two row and column
 object orderings given by the final permutations \verb+OUTPERMONE+
 and \verb+OUTPERMTWO+. As before, \verb+KBLOCK+ defines the block size in the use  of the
 iterative quadratic assignment routine.


 \tiny

\begin{verbatim}
  [find,vaf,targone,targtwo,outpermone,outpermtwo] = ...
bicirarobfnd(number,randperm(10),1)

find =

  Columns 1 through 6

         0    0.4210    0.5632    0.7297    0.6840    0.8040
    0.4210         0    0.3048    0.3252    0.6460    0.5880
    0.5632    0.3048         0    0.3540    0.0380    0.6535
    0.7297    0.3252    0.3540         0    0.4340    0.4154
    0.6840    0.6460    0.0380    0.4340         0    0.4401
    0.8040    0.5880    0.6535    0.4154    0.4401         0
    0.7871    0.7580    0.4208    0.3317    0.3565    0.3963
    0.9090    0.6380    0.8131    0.5750    0.7418    0.4000
    0.8210    0.7926    0.3881    0.7841    0.2460    0.6710
    0.8521    0.6380    0.7841    0.2631    0.6830    0.5899

  Columns 7 through 10

    0.7871    0.9090    0.8210    0.8521
    0.7580    0.6380    0.7926    0.6380
    0.4208    0.8131    0.3881    0.7841
    0.3317    0.5750    0.7841    0.2631
    0.3565    0.7418    0.2460    0.6830
    0.3963    0.4000    0.6710    0.5899
         0    0.4176    0.3500    0.2960
    0.4176         0    0.4000    0.4590
    0.3500    0.4000         0    0.3920
    0.2960    0.4590    0.3920         0


vaf =

    0.9955


targone =

  Columns 1 through 6

         0    0.0858    0.0858    0.3086    0.4576    0.4576
    0.0858         0    0.0096    0.2443    0.2443    0.3863
    0.0858    0.0096         0    0.2133    0.2391    0.2391
    0.3086    0.2443    0.2133         0    0.0994    0.1207
    0.4576    0.2443    0.2391    0.0994         0    0.1207
    0.4576    0.3863    0.2391    0.1207    0.1207         0
    0.4818    0.4818    0.3631    0.2195    0.2195    0.2195
    0.4818    0.4818    0.4818    0.2195    0.2195    0.2195
    0.3153    0.4902    0.4902    0.4902    0.4711    0.3356
    0.3153    0.4361    0.4628    0.4902    0.7370    0.7185

  Columns 7 through 10

    0.4818    0.4818    0.3153    0.3153
    0.4818    0.4818    0.4902    0.4361
    0.3631    0.4818    0.4902    0.4628
    0.2195    0.2195    0.4902    0.4902
    0.2195    0.2195    0.4711    0.7370
    0.2195    0.2195    0.3356    0.7185
         0   -0.0393    0.3356    0.4818
   -0.0393         0    0.3356    0.4818
    0.3356    0.3356         0    0.2371
    0.4818    0.4818    0.2371         0


targtwo =

  Columns 1 through 6

         0    0.0765    0.1367    0.2969    0.2969    0.2969
    0.0765         0    0.0289    0.2395    0.3704    0.3704
    0.1367    0.0289         0    0.1609    0.3582    0.4319
    0.2969    0.2395    0.1609         0    0.1905    0.2793
    0.2969    0.3704    0.3582    0.1905         0    0.0670
    0.2969    0.3704    0.4319    0.2793    0.0670         0
    0.2678    0.3024    0.3024    0.3024    0.1839    0.1169
    0.1122    0.3024    0.3024    0.3555    0.2480    0.1959
    0.1122    0.3024    0.3024    0.3555    0.2480    0.1959
    0.1122    0.2012    0.2364    0.3555    0.2480    0.1959

  Columns 7 through 10

    0.2678    0.1122    0.1122    0.1122
    0.3024    0.3024    0.3024    0.2012
    0.3024    0.3024    0.3024    0.2364
    0.3024    0.3555    0.3555    0.3555
    0.1839    0.2480    0.2480    0.2480
    0.1169    0.1959    0.1959    0.1959
         0   -0.0105   -0.0105    0.1558
   -0.0105         0   -0.1278   -0.0478
   -0.0105   -0.1278         0   -0.0478
    0.1558   -0.0478   -0.0478         0


outpermone =

     3     5     9     7     6     8    10     4     2     1


outpermtwo =

     7    10     9     8     1     6     2     3     4     5
\end{verbatim}


\large

For finding multiple CSAR forms,  \verb+bicirsarobfnd.m+ has usage syntax
\begin{verbatim}
[find,vaf,targone,targtwo,outpermone,outpermtwo] = ...
bicirsarobfnd(prox,inperm,kblock)
\end{verbatim}
\noindent with all the various terms the same as for \verb+bicirarobfnd.m+ but now for strongly CAR (CSAR) structures.  The example below finds essentially the same representation as above (involving digit magnitude and structure) with a slight drop in the \verb+VAF+ from 99.55\% for CAR to 91.06\% for CSAR.

\tiny

\begin{verbatim}
  [find,vaf,targone,targtwo,outpermone,outpermtwo] = ...
bicirsarobfnd(number,randperm(10),1)

find =

  Columns 1 through 6

         0    0.4212    0.6464    0.6464    0.6840    0.8040
    0.4212         0    0.3284    0.3284    0.5122    0.6693
    0.6464    0.3284         0    0.3273    0.0947    0.6682
    0.6464    0.3284    0.3273         0    0.5111    0.3505
    0.6840    0.5122    0.0947    0.5111         0    0.4090
    0.8040    0.6693    0.6682    0.3505    0.4090         0
    0.8420    0.7215    0.4802    0.4027    0.4493    0.3718
    0.8420    0.7215    0.7215    0.6565    0.6906    0.4000
    0.8420    0.7215    0.3041    0.7204    0.2732    0.6895
    0.8420    0.7215    0.7204    0.2630    0.6895    0.5540

  Columns 7 through 10

    0.8420    0.8420    0.8420    0.8420
    0.7215    0.7215    0.7215    0.7215
    0.4802    0.7215    0.3041    0.7204
    0.4027    0.6565    0.7204    0.2630
    0.4493    0.6906    0.2732    0.6895
    0.3718    0.4000    0.6895    0.5540
         0    0.4055    0.2292    0.3339
    0.4055         0    0.4705    0.4055
    0.2292    0.4705         0    0.4694
    0.3339    0.4055    0.4694         0


vaf =

    0.9106


targone =

  Columns 1 through 6

         0    0.3924    0.6326    0.6326    0.6326    0.6337
    0.3924         0    0.3149    0.3149    0.4970    0.5686
    0.6326    0.3149         0    0.3149    0.4970    0.5686
    0.6326    0.3149    0.3149         0    0.1752    0.5686
    0.6326    0.4970    0.4970    0.1752         0    0.5686
    0.6337    0.5686    0.5686    0.5686    0.5686         0
    0.6337    0.6337    0.6337    0.6337    0.6337    0.6337
    0.6337    0.6337    0.6337    0.6337    0.6337    0.6337
    0.2162    0.3924    0.6326    0.6326    0.6326    0.6337
    0.2162    0.3924    0.6326    0.6326    0.6326    0.6337

  Columns 7 through 10

    0.6337    0.6337    0.2162    0.2162
    0.6337    0.6337    0.3924    0.3924
    0.6337    0.6337    0.6326    0.6326
    0.6337    0.6337    0.6326    0.6326
    0.6337    0.6337    0.6326    0.6326
    0.6337    0.6337    0.6337    0.6337
         0    0.4085    0.6337    0.6337
    0.4085         0    0.6337    0.6337
    0.6337    0.6337         0    0.2162
    0.6337    0.6337    0.2162         0


targtwo =

  Columns 1 through 6

         0   -0.2236    0.0570    0.0570    0.0570    0.0570
   -0.2236         0   -0.1686    0.0570    0.0570    0.0570
    0.0570   -0.1686         0   -0.1632   -0.1632   -0.1632
    0.0570    0.0570   -0.1632         0   -0.1632   -0.1632
    0.0570    0.0570   -0.1632   -0.1632         0   -0.1632
    0.0570    0.0570   -0.1632   -0.1632   -0.1632         0
    0.0503    0.1703    0.2083    0.2083    0.2083    0.2083
   -0.1215    0.0356    0.0878    0.0878    0.0878    0.0878
   -0.1215    0.0356    0.0878    0.0878    0.0878    0.0878
   -0.1215    0.0356    0.0878    0.0878    0.0878    0.0878

  Columns 7 through 10

    0.0503   -0.1215   -0.1215   -0.1215
    0.1703    0.0356    0.0356    0.0356
    0.2083    0.0878    0.0878    0.0878
    0.2083    0.0878    0.0878    0.0878
    0.2083    0.0878    0.0878    0.0878
    0.2083    0.0878    0.0878    0.0878
         0    0.0127    0.0127    0.0127
    0.0127         0   -0.3053   -0.3053
    0.0127   -0.3053         0   -0.3053
    0.0127   -0.3053   -0.3053         0


outpermone =

     5     7     6     4    10     8     2     1     9     3


outpermtwo =

     5     6     8     9    10     7     1     4     3     2
\end{verbatim}


\large

\section{Anti-Robinson (AR) Matrices for Two-Mode  Proximity Data}

In direct analogy to the extensions of Linear Unidimensional Scaling (LUS), it is possible to find and fit (more general) anti-Robinson (AR) forms to two-mode proximity matrices.  The same type of reordering strategy implemented by \verb+ordertm.m+ would be used, but the more general AR form would be fitted to the reordered square proximity matrix, $\mathbf{P}^{(tm)}_{\rho_{0}} = \{p^{(tm)}_{\rho_{0}(i) \rho_{0}(j)}\}$; the least-squares criterion
\[ \sum_{i,j = 1}^{n} w_{\rho_{0}(i) \rho_{0}(j)}(p^{(tm)}_{\rho_{0}(i) \rho_{0}(j)} - \hat{p}_{ij})^{2} , \]
is minimized, where  $w_{\rho_{0}(i) \rho_{0}(j)} = 0$ if $\rho_{0}(i)$ and $\rho_{0}(j)$ are both row or both column objects, and $= 1$ otherwise.  The entries in the matrix $\{\hat{p}_{ij}\}$ fitted to $\mathbf{P}^{(tm)}_{\rho_{0}}$ are AR in form (and which correspond to nonzero values of the weight function $w_{\rho_{0}(i) \rho_{0}(j)}$), and thus satisfy certain linear inequality constraints generated from how the row and column objects are intermixed by the given permutation $\rho_{0}(\cdot)$.   We note here and discuss this more completely in the section to follow that the patterning of entries in $\{\hat{p}_{ij}\}$  fitted to the original two-mode proximity matrix, with appropriate row and column permutations extracted from $\rho_{0}$, is called an anti-Q-form.

\subsection{Fitting and Finding Two-Mode AR Matrices}

The M-file \verb+arobfittm.m+ does a confirmatory two-mode anti-Robinson fitting of a given
ordering of the row and column objects of a two-mode proximity matrix
using the Dykstra-Kaczmarz iterative projection least-squares method.  The usage syntax has the form
\begin{verbatim}
[fit,vaf,rowperm,colperm] = arobfittm(proxtm,inperm)
\end{verbatim}
\noindent where \verb+PROXTM+ is the input two-mode proximity matrix; \verb+INPERM+ is the given ordering of the row and column objects together; \verb+FIT+  is an $n_{a} \times n_{b}$ (number of rows by number of columns) matrix
fitted to \verb+PROXTM(ROWPERM,COLPERM)+ with \verb+VAF+ being the variance-accounted-for
based on the (least-squares criterion) sum of
squared discrepancies between \verb+PROXTM(ROWPERM,COLMEAN)+ and \verb+FIT+; \verb+ROWPERM+ and \verb+COLPERM+ are the row and column object orderings derived from \verb+INPERM+.

\smallskip

The matrix given by \verb+FIT+ that is intended to approximate the row and column permuted two-mode proximity matrix, \verb+PROXTM(ROWPERM,COLPERM)+, displays a particularly important patterning of its entries called an anti-Q-form in the literature (see Hubert and Arabie, 1995,   for an extended discussion of this type of patterning for a two-mode matrix).  Specifically, a matrix is said to have the anti-Q-form (for rows and columns) if within each row and column the entries are nonincreasing to a minimum and thereafter nondecreasing.  \emph{Matrices satisfying the anti-Q-form have a convenient interpretation presuming an underlying unidimensional scale that jointly represents both the row and column objects}.   Explicitly, suppose a matrix has been appropriately row-ordered to display the anti-Q-form for columns.  Any dichotomization of the entries within a column at some threshold value (using 0 for entries below the threshold and 1 if at or above), produces a matrix that has the consecutive zeros property within each column, that is, all zeros within a column occur consecutively, uninterrupted by intervening ones.  In turn, any matrix with the consecutive zeros property for columns suggests the existence of a perfect scale (error-free), where row objects can be ordered along a continuum (using the same row order for the matrix that actually reflects the anti-Q-form for columns), and each column object is representable as an interval along the continuum (encompassing those consecutive row objects corresponding to zeros).
Historically, the type of pattern represented by the anti-Q-form has played a major role in the literature of (unidimensional) unfolding, and for example, is the basis of Coombs's (1964, Chapter 4)  parallelogram structure for a two-mode proximity matrix.  The reader is referred to Hubert (1974) for a review of some of these connections.

\smallskip


To provide an example of what an anti-Q-form looks like for our two-mode data matrix, \verb+goldfish_receptor+, we will use \verb+arobfndtm.m+  both to find and fit an anti-Robinson form using iterative projection to
a two-mode proximity matrix in the $L_{2}$-norm based on a permutation
identified through the use of iterative quadratic assignment. The usage syntax is
\begin{verbatim}
[find, vaf, outperm, rowperm, colperm] = ...
 arobfndtm(proxtm, inperm, kblock)
\end{verbatim}
\noindent where, again, \verb+INPERM+ is a given starting permutation of the first $n = n_{a} + n_{b}$ integers;
\verb+FIND+ is the least-squares optimal matrix (with variance-accounted-for of \verb+VAF+) displaying an anti-Q-form (because of the anti-Robinson form constructed for the combined row and column object ordering given by the ending permutation \verb+OUTPERM+). \verb+KBLOCK+ defines the block size in the use the iterative quadratic assignment
routine. \verb+ROWPERM+ and \verb+COLPERM+ are the resulting row and column permutations for
the objects.  In the listing below, the \verb+VAF+ for the given fitted matrix is very high: .9667 (which can be compared to the alternative representations given earlier with values of .8072 (linear unidimensional scaling),
.6209 (ultrametric), and .8663 (additive tree)).


\tiny

\begin{alltt}
  load goldfish_receptor.dat
  [find,vaf,outperm,rowperm,colperm] = ...
arobfndtm(goldfish_receptor,randperm(20),2);
  find

find =

  Columns 1 through 6

   68.0000   54.5000   80.0000  138.0000  145.0000  162.8000
   71.5000   54.5000   64.0000  128.0000  144.0000  162.8000
   71.5000   47.0000   61.0000  117.5000  117.5000  145.0000
   80.0000   47.5000   47.5000   98.0000  116.0000  137.5000
  155.0000  108.0000   63.0000   94.0000  103.0000  137.5000
  174.0000  125.0000   84.0000   49.0000   47.6667   76.0000
  200.0000  143.0000   91.0000   49.0000   47.6667   76.0000
  200.0000  156.0000  107.0000   67.0000   47.6667   60.0000
  200.0000  183.0000  177.0000  176.0000  168.0000  112.5000
  200.0000  200.0000  200.0000  198.0000  186.0000  112.5000
  200.0000  200.0000  200.0000  198.0000  188.0000  143.0000

  Columns 7 through 9

  162.8000  200.0000  200.0000
  162.8000  162.8000  173.0000
  145.0000  151.6667  158.0000
  138.5000  151.6667  158.0000
  138.5000  151.6667  158.0000
  106.0000  134.5000  134.5000
  106.0000  124.5000  124.5000
   78.0000  100.0000  100.0000
   82.5000   47.0000   46.0000
   82.5000   54.0000   47.5000
  111.0000   54.0000   47.5000

  vaf

vaf =

    0.9667

  outperm

outperm =

  Columns 1 through 10

    20    11    10    19     9    18     8     7    17    16

  Columns 11 through 20

     6     5     4    15    14    13     3    12     2     1

  rowperm\textsf{'}

ans =

  Columns 1 through 10

    11    10     9     8     7     6     5     4     3     2

  Column 11

     1

  colperm\textsf{'}

ans =

     9     8     7     6     5     4     3     2     1
\end{alltt}

\large

\subsection{Multiple Two-Mode AR Reorderings and Fittings}

The M-file, \verb+biarobfndtm.m+, finds and fits the sum of two anti-Q-forms (extracted from fitting two anti-Robinson matrices)  using iterative projection to
a two-mode proximity matrix in the $L_{2}$-norm based on permutations
identified through the use of iterative quadratic assignment.  In the usage
\begin{verbatim}
[find,vaf,targone,targtwo,outpermone,outpermtwo, ...
        rowpermone,colpermone,rowpermtwo,colpermtwo] = ...
    biarobfndtm(proxtm,inpermone,inpermtwo,kblock)
\end{verbatim}
\noindent
\verb+PROXTM+ is the usual input two-mode proximity matrix ($n_{a} \times n_{b}$)
with a dissimilarity interpretation, and
\verb+FIND+ is the least-squares optimal matrix (with variance-accounted-for
of \verb+VAF+) to \verb+PROXTM+. The latter matrix \verb+PROXTM+ is the sum of the two  matrices
\verb+TARGONE+ and \verb+TARGTWO+ based on the two row and column
object orderings given by the ending permutations \verb+OUTPERMONE+
and \verb+OUTPERMTWO+.  The two ending permutations of \verb+OUTPERMONE+
and \verb+OUTPERMTWO+ contain the ending row and column object orderings of \verb+ROWPERMONE+ and \verb+ROWPERMTWO+ and \verb+COLPERMONE+
and \verb+COLPERMTWO+. \verb+KBLOCK+ defines the block size in the use the
iterative quadratic assignment routine; the input permutations are \verb+INPERMONE+ and
\verb+INPERMTWO+.

\smallskip

As can be seen in the example below, the sum of two anti-Q-forms fitted to the \verb+goldfish_receptor+ data provides an almost perfect reconstruction (with a \verb+VAF+ of .9995).

\tiny

\begin{alltt}
  [find,vaf,targone,targtwo,outpermone,outpermtwo, ...
rowpermone,colpermone,rowpermtwo,colpermtwo] = ...
biarobfndtm(goldfish_receptor,randperm(20),randperm(20),2);

find

find =

  Columns 1 through 6

   47.3504   54.6226  111.0000  143.0000  188.0000  196.1177
   47.3504   54.6226   75.0000  100.0000  186.0000  199.8954
   46.0072   47.1738   90.0072  124.9856  167.9981  175.9981
   99.0098  101.1738   78.0098   59.9508   46.0000   66.9608
  122.0098  127.1738  115.0098   79.0098   48.9981   47.8673
  116.6197  152.0595   96.5053   72.7463   48.0000   52.0523
  198.0000  186.0000  154.0000  148.0000  103.0000   94.0000
  133.8750  156.0000  123.0000  126.7482  115.0000   98.0523
  141.0000  113.0000  142.0000  145.4795  115.0000  121.9575
  173.0000  140.0000  176.5991  176.4009  144.6719  128.6719
  200.2992  200.1738  160.7486  160.7486  145.0000  138.0000

  Columns 7 through 9

  199.9608  200.0000  201.9327
  200.0000  199.9924  197.3046
  176.9981  182.9981  200.0135
  106.9608  156.0000  200.0000
   89.1327  142.9981  199.9981
   84.0000  124.9924  173.3897
   63.0000  108.0000  155.0000
   49.0000   46.0153   79.3897
   60.0425   47.0000   60.9419
   66.5870   56.6719   84.3973
   80.0000   53.0000   65.5700

vaf

vaf =

    0.9995

targone

targone =

  Columns 1 through 6

   46.1875   46.1875  111.0900  143.0900  189.1858  197.1531
   46.1875   46.1875   83.7500  118.2801  187.1858  197.1531
   39.3017   38.7387   83.3017  118.2801  168.0431  176.0431
   96.3403   92.7387   75.3403   57.2813   57.2813   67.9962
  119.3403  118.7387  112.3403   76.3403   49.0431   49.7778
  143.6244  143.6244  123.5100   80.9988   49.1858   49.3100
  159.4691  147.4691  138.5000  135.0006  102.8519   94.0000
  160.8797  151.8810  138.5000  135.0006  116.1858   95.3100
  160.8797  153.7319  153.7319  153.7319  116.1858  113.2992
  173.0000  166.2124  164.3730  164.1748  129.4690  113.4690
  199.1362  191.7387  164.3730  164.3730  146.1858  138.0000

  Columns 7 through 9

  200.9962  203.4720  205.5571
  200.0000  202.1911  205.5571
  177.0431  183.0431  200.0585
  107.9962  156.7769  200.0585
   91.0431  143.0431  200.0431
   84.0000  127.1911  181.6422
   63.0000  107.8519  154.8519
   48.2140   48.2140   87.6422
   51.3841   48.2140   69.1944
   51.3841   41.4690   69.1944
   80.0000   55.2081   69.1944

targtwo

targtwo =

  Columns 1 through 6

  -26.2124         0   12.2261   12.2261   15.2029   15.2029
  -40.7319  -19.8797  -11.7319   -8.2525   -8.2525   -1.2140
    4.1190  -27.0047  -15.5000   -8.2525   -8.2525   -2.1988
    8.4351  -27.0047  -27.0047   -8.2525   -8.2525   -2.1988
    8.4351    1.1630   -8.7500  -18.2801   -8.2525   -2.1988
    8.4351    1.1630   -3.6244   -3.6244   -3.6244   -2.2081
    8.4351    1.1630   -0.0900   -0.0900   -3.6244   -3.4720
    8.4351    2.6696    2.6696    2.6696   -0.0585   -0.7769
    8.4351    2.6696    2.6696    2.6696   -0.0450   -0.0450
    8.4351    6.7055    6.7055    6.7055   -0.0450   -0.0450
   38.5309   38.5309   15.5000   12.9994    0.1481    0.1481

  Columns 7 through 9

   15.2029   15.2029   15.2029
   -1.1858    8.6584    8.6584
   -1.1858    0.7860    2.7423
   -1.1858         0    2.7423
   -1.1858         0    2.7423
   -1.1858         0         0
   -1.1858   -1.0354   -1.0354
  -11.2813   -1.0354   -1.0354
   -0.0450   -1.9104   -1.9104
   -0.0450   -0.0450   -0.0450
    0.1481         0         0

outpermone

outpermone =

  Columns 1 through 10

     1     2    12    13     3    14     4    15     5    16

  Columns 11 through 20

     6    17     7     8    18     9    10    19    11    20

outpermtwo

outpermtwo =

  Columns 1 through 10

    10    13     9    12     8    14     6    15     2    20

  Columns 11 through 20

    11     1    19    16     4     5    18    17     3     7

rowpermone\textsf{'}

ans =

  Columns 1 through 10

     1     2     3     4     5     6     7     8     9    10

  Column 11

    11

colpermone\textsf{'}

ans =

     1     2     3     4     5     6     7     8     9

rowpermtwo\textsf{'}

ans =

  Columns 1 through 10

    10     9     8     6     2    11     1     4     5     3

  Column 11

     7

colpermtwo\textsf{'}

ans =

     2     1     3     4     9     8     5     7     6
\end{alltt}


\large


\section{An Open-Source Mixed Integer Linear Programming Routine for MATLAB}

The MATLAB Optimization Toolbox includes (a fairly recent) routine for binary integer programming problems.  It solves tasks of the form: $\min_{\mathbf{x}} \mathbf{f}'\mathbf{x}$, such that $\mathbf{A}\mathbf{x} \le \mathbf{b}$ and $\mathbf{A}_{eq}\mathbf{x} = \mathbf{b}_{eq}$, where the entries in the vector $\mathbf{x}$ can take on only the values of 1 or 0; $\mathbf{b}$, $\mathbf{b}_{eq}$, and $\mathbf{f}$ are vectors; $\mathbf{A}$ and $\mathbf{A}_{eq}$ are matrices defining the inequality and equality constraints, respectively.

\smallskip

A more general optimization routine is the open-source routine, \verb+lp_solve.m+, for mixed integer linear programming, where some of the decision variables may not be integer, and the integer variables are not restricted to be binary.  We give the help header below, which should be sufficient to formulate most applications of it we envision:

\tiny

\begin{verbatim}
LP_SOLVE  Solves mixed integer linear programming problems.

   SYNOPSIS: [obj,x,duals,stat] = lp_solve(f,a,b,e,vlb,vub,xint,scalemode,keep)

      solves the MILP problem

              max v = f'*x
                a*x <> b
                  vlb <= x <= vub
                  x(int) are integer

   ARGUMENTS: The first four arguments are required:

            f: n vector of coefficients for a linear objective function.
            a: m by n matrix representing linear constraints.
            b: m vector of right sides for the inequality constraints.
            e: m vector that determines the sense of the inequalities:
                      e(i) = -1  ==> Less Than
                      e(i) =  0  ==> Equals
                      e(i) =  1  ==> Greater Than
          vlb: n vector of lower bounds. If empty or omitted,
               then the lower bounds are set to zero.
          vub: n vector of upper bounds. May be omitted or empty.
         xint: vector of integer variables. May be omitted or empty.
    scalemode: scale flag. Off when 0 or omitted.
         keep: Flag for keeping the lp problem after it's been solved.
               If omitted, the lp will be deleted when solved.

   OUTPUT: A nonempty output is returned if a solution is found:

          obj: Optimal value of the objective function.
            x: Optimal value of the decision variables.
        duals: solution of the dual problem.
\end{verbatim}

\large

\noindent As an example, suppose we wish to maximize: $-x_{1} + 2x_{2}$, subject to: $2x_{1} + x_{2} < 5$ and $-4x_{1} + 4x_{2} < 5$, with both $x_{1}$ and $x_{2}$ integer:

\tiny
\begin{verbatim}
>> [obj,x] = lp_solve([-1,2],[2,1;-4,4],[5,5],[-1,-1],[],[],[1,2])

obj =

     3


x =

     1
     2
 \end{verbatim}

 \large

 \noindent If it is desired to relax $x_{1}$ and allow it to be anything, the vector, \verb+xint+, is changed from \verb+[1,2]+ to just \verb+[2]+; if we relax both, \verb+xint+ is changed to \verb+[]+.

 \tiny

 \begin{verbatim}

>> [obj,x] = lp_solve([-1,2],[2,1;-4,4],[5,5],[-1,-1],[],[],[2])

obj =

    3.2500


x =

    0.7500
    2.0000

>> [obj,x] = lp_solve([-1,2],[2,1;-4,4],[5,5],[-1,-1],[],[],[])

obj =

    3.7500


x =

    1.2500
    2.5000
\end{verbatim}

\large

\section{The Analysis of Skew-Symmetric Proximity Matrices}

Suppose the original proximity information given between the pairs of objects in $S = \{O_{1},\ldots,O_{n}\}$ is nonsymmetric and in the form of an $n \times n$ nonnegative matrix, say $\mathbf{D} = \{d_{ij}\}$.  This latter matrix will be decomposed into the sum of its symmetric and skew-symmetric components as \[ \mathbf{D} = [(\mathbf{D} + \mathbf{D}')/2] + [(\mathbf{D} - \mathbf{D}')/2] \ , \]
\noindent and each of these components will always be addressed separately.  The matrix $(\mathbf{D} + \mathbf{D}')/2$ is a nonnegative symmetric matrix (in our notation, $\mathbf{P}$), and all the methods appropriate for such a proximity measure can be applied.  The second component, $(\mathbf{D} - \mathbf{D}')/2$, is an $n \times n$ matrix denoted by $\mathbf{P}^{ss} = \{p_{ij}^{ss}\}$, where $p_{ij}^{ss} = (d_{ij} - d_{ji})/2$ for $1 \le i,j \le n$, and the superscript `ss' signifies `skew-symmetric', i.e., $p_{ij}^{ss} = -p_{ji}^{ss}$ for all $1 \le i,j \le n$.  Thus, there is an explicit directionality to the (dominance) relationship specified between any two objects depending on the order of the subscripts, i.e., $p_{ij}^{ss}$ and  $p_{ji}^{ss}$ are equal in absolute magnitude but differ in sign.  It may also be worth noting here that any skew-symmetric matrix $\{p_{ij}^{ss}\}$ can itself be interpreted as containing two separate sources of information: one is the directionality of the dominance relationship given by $\{\mathrm{sign}(p_{ij}^{ss})\}$, where $\mathrm{sign}(y) = 1$ if $y > 0$; $=0$ if $y = 0$; and $= -1$ if $y < 0$; the second is in the absolute magnitude of dominance given by $\{\mid p_{ij}^{ss} \mid\}$.  This latter matrix is symmetric and can be viewed as a dissimilarity matrix and analyzed as such.

\smallskip

To eventually have (two) skew-symmetric matrices to use in our examples, we begin with a very old data set, originally collected in 1929 in a study of the influence of motion pictures on children's attitudes (see Thurstone, 1959, pp.\ 309--319).  Both before and after seeing a film entitled \emph{Street of Chance}, which depicted the life of a gambler, 240 school children were asked to compare the relative seriousness of thirteen offenses presented in all 78 possible pairs:

\bigskip

1): bankrobber

2): gambler

3): pickpocket

4): drunkard

5): quack doctor

6): bootlegger

7): beggar

8): gangster

9): tramp

10): speeder

11): petty thief

12): kidnaper

13): smuggler

\bigskip

\noindent The data are given below, where the entries show the proportion of the school children who rated the offense listed in the column to be more serious than the offense listed in the row.  The above-diagonal entries were obtained before the showing of the film; those below were collected after. The obvious substantive question here involves the effect of the film on the assessment of the offense of being a gambler.

\bigskip

\begin{tabular}{llllllllllllll}




 1:bankrobber    & x   & .07  & .08  & .05  & .27  & .29  & .01  & .50  & .00  & .06 & .02  & .73  & .21   \\
 2:gambler    & .79   & x  & .71  & .52  & .76  & .92  & .07  & .92  & .05  & .41 &  .49 & .90  & .81   \\
 3:pickpocket    & .93   & .51  & x  & .25  & .67  & .75  & .02  & .86  & .02  & .39 & .42  & .87  & .68   \\
 4:drunkard    &  .95  & .70  & .70  & x  & .81  & .95  & .01  & .92  & .03  & .37 & .62  & .91  & .87   \\
 5:quack doctor    &  .67  & .36  & .28  & .16  &  x  & .49  & .02  & .70  & .02  & .12 & .22  & .64  & .55   \\
 6:bootlegger    & .70   & .31  & .30  & .13  & .50  & x  & .00  & .79  & .01  & .09 & .26  & .68  & .50   \\
 7:beggar    &  .98  & .95  & .97  & .94  & .98  & .98  & x  & .96  & .42  & .86 & .96  & 1.0  & .99   \\
 8:gangster    & .50   & .18  & .13  & .11  & .32  & .27  & .01  & x  & .02  & .08 & .08  & .36  & .31   \\
 9:tramp    &  1.0  & .96  & .98  & .96  & .99  & .98  & .64  & .99  & x  & .91 & .97  & .99  & 1.0   \\
 10:speeder    & .94   & .73  & .68  & .67  & .89  & .90  & .21  & .94  & .13  & x & .58  & .90  & .92   \\
 11:petty thief    & .97   & .64  & .62  & .47  & .81  & .76  & .06  & .89  & .05  & .36 & x  & .98  & .78   \\
 12:kidnaper    & .38   & .27  & .16  & .08  & .35  & .30  & .02  & .62  & .01  & .08 & .03  & x  & .27   \\
 13:smuggler    &  .73  & .31  & .30  & .16  & .46  & .49  & .02  & .66  & .02  & .11 & .24  & .64  & x   \\



  \end{tabular}

  \bigskip

  To generate two skew-symmetric matrices from the data just given, the entry defined for each pair of offenses in the `before' and `after' skew-symmetric matrices, is a difference in the proportions of rating one offense more serious than the other.  For example, because the proportion judging a bootlegger more serious than a bankrobber is .29 before the movie was shown, an entry of $.29 - .79 = -.42$ is present for the (bankrobber,  bootlegger) above-diagonal pair in the before matrix: the negative value above the diagonal indicates that the second member (bootlegger) is less serious than the first (bankrobber).  The two data matrices are called \verb+thurstone_skew_symmetric_before.dat+ and   \verb+thurstone_skew_symmetric_after.dat+, and have verbatim contents as follow:

  \bigskip

  \

  \verb+thurstone_skew_symmetric_before.dat+ :

  \tiny
  \begin{verbatim}
   .00  -.86  -.84  -.90  -.46  -.42  -.98   .00 -1.00  -.88  -.96   .46  -.58
   .86   .00   .42   .04   .52   .84  -.86   .84  -.90  -.18  -.02   .80   .62
   .84  -.42   .00  -.50   .34   .50  -.96   .72  -.96  -.22  -.16   .74   .36
   .90  -.04   .50   .00   .62   .90  -.98   .84  -.94  -.26   .24   .82   .74
   .46  -.52  -.34  -.62   .00  -.02  -.96   .40  -.96  -.76  -.56   .28   .10
   .42  -.84  -.50  -.90   .02   .00 -1.00   .58  -.98  -.82  -.48   .36   .00
   .98   .86   .96   .98   .96  1.00   .00   .92  -.16   .72   .92  1.00   .98
   .00  -.84  -.72  -.84  -.40  -.58  -.92   .00  -.97  -.84  -.84  -.28  -.38
  1.00   .90   .96   .94   .96   .98   .16   .97   .00   .82   .94   .98  1.00
   .88   .18   .22   .26   .76   .82  -.72   .84  -.82   .00   .16   .80   .84
   .96   .02   .16  -.24   .56   .48  -.92   .84  -.94  -.16   .00   .96   .56
  -.46  -.80  -.74  -.82  -.28  -.36 -1.00   .28  -.98  -.80  -.96   .00  -.46
   .58  -.62  -.36  -.74  -.10   .00  -.98   .38 -1.00  -.84  -.56   .46   .00
   \end{verbatim}

   \large
   \bigskip




  \verb+thurstone_skew_symmetric_after.dat+ :

  \tiny
  \begin{verbatim}
   .00  -.58  -.86  -.90  -.34  -.40  -.96   .00 -1.00  -.88  -.94   .24  -.46
   .58   .00  -.02  -.40   .28   .38  -.90   .64  -.92  -.46  -.28   .46   .38
   .86   .02   .00  -.40   .44   .40  -.94   .74  -.96  -.36  -.24   .68   .40
   .90   .40   .40   .00   .68   .74  -.88   .78  -.92  -.34   .06   .84   .68
   .34  -.28  -.44  -.68   .00   .00  -.96   .36  -.98  -.78  -.62   .30   .08
   .40  -.38  -.40  -.74   .00   .00  -.96   .46  -.96  -.80  -.52   .40   .02
   .96   .90   .94   .88   .96   .96   .00   .98  -.28   .58   .88   .96   .96
   .00  -.64  -.74  -.78  -.36  -.46  -.98   .00  -.98  -.88  -.78  -.24  -.32
  1.00   .92   .96   .92   .98   .96   .28   .98   .00   .74   .90   .98   .96
   .88   .46   .36   .34   .78   .80  -.58   .88  -.74   .00   .28   .84   .78
   .94   .28   .24  -.06   .62   .52  -.88   .78  -.90  -.28   .00   .94   .52
  -.24  -.46  -.68  -.84  -.30  -.40  -.96   .24  -.98  -.84  -.94   .00  -.28
   .46  -.38  -.40  -.68  -.08  -.02  -.96   .32  -.96  -.78  -.52   .28   .00
\end{verbatim}

\large

\bigskip

Measures of matrix patterning for a skew-symmetric matrix, $\mathbf{P}^{ss}$, that might be derivable indirectly from the generation of some type of coordinate representation, have an interestingly different status than they do for a symmetric matrix $\mathbf{P}$.  In the skew-symmetric framework, closed-form least-squares solutions are possible, thus eliminating the need for some auxiliary optimization strategy.  For example, suppose we wish to find a set of $n$ coordinate values, $x_{1},\ldots,x_{n}$, such that the least-squares criterion,
 \[ \sum_{i < j} (p_{ij}^{ss} - (x_{j} - x_{i}))^{2} \ , \]
 \noindent is minimized.  An optimal set of coordinates can be obtained analytically from the average proximity within column $j$, i.e., $x_{j} = (1/n)\sum_{i} p_{ij}^{ss}$, with a minimum loss value of
 \[ \sum_{i < j} (p_{ij}^{ss})^{2} - (1/n) \sum_{j} (\sum_{i} p_{ij}^{ss})^{2} \ . \]
 \noindent Also, because the sum of the entries in $\mathbf{P}^{ss}$ is zero, the variance-accounted-for (vaf) can be given simply as:
 \[ \mathrm{vaf} = \frac{(1/n) \sum_{j} (\sum_{i} p_{ij}^{ss})^{2}}{ \sum_{i < j} (p_{ij}^{ss})^{2}} \ . \]
 \noindent Thus, an optimal row/column reordering of $\mathbf{P}^{ss}$ can be obtained just by using the order of the optimal coordinates from smallest (most negative) to largest (most positive).  Similarly, if we consider an equally-spaced coordinate representation obtained by minimizing the least-squares loss function:
  \[ \sum_{i < j} (p_{ij}^{ss} - \alpha (x_{j} - x_{i}))^{2} \ , \]
  \noindent where $x_{1},\ldots,x_{n}$ are the integers $1,\ldots,n$ in some order, and $\alpha$ is some multiplicative constant to be estimated, the optimal row/column reordering of $\mathbf{P}^{ss}$ induced by the integer coordinates would again be generated by the ordering of $(1/n) \sum_{i} p_{ij}^{ss}$ for $1 \le j \le n$.

  \smallskip

  The M-file we have written to carry-out these closed-form scaling procedures on a skew-symmetric proximity matrix is \verb+skew_symmetric_scaling.m+, with syntax:
  \begin{verbatim}
  [coord,sort_coord,permut,prox_permut,vaf,...
    alpha_multiplier,alpha_vaf,alpha_coord] = ...
    skew_symmetric_scaling(prox)
\end{verbatim}
\noindent Here,  \verb+COORD+ contains the least-squares coordinates which are sorted from
  smallest to largest in \verb+SORT_COORD+; \verb+VAF+ is the variance they account for;
  \verb+PERMUT+ is the object permutation
  corresponding to \verb+SORT_COORD+ with \verb+PROX_PERMUT+ the reordered skew-symmetric
  proximity matrix.  For equally-spaced coordinates, \verb+ALPHA_MULTIPLIER+
  defines the multiplicative constant on the integer-valued coordinates; a
  collection of equally-spaced coordinates is given by \verb+ALPHA_COORD+ with
  \verb+ALPHA_VAF+ the variance they account for.

  \smallskip

  The two Thurstone skew-symmetric matrices are used in the MATLAB recording below.  One can see the `gambler' offense moving from a fifth to a seventh position in the `before' and `after' analyses.  It does appears that the movie has had an effect.

  \tiny
  \begin{verbatim}
  >> load thurstone_skew_symmetric_before.dat
>> load thurstone_skew_symmetric_after.dat

>> [coord,sort_coord,permut,prox_permut,vaf,alpha_multiplier,alpha_vaf,...
alpha_coord] = skew_symmetric_scaling(thurstone_skew_symmetric_before)

coord =

    0.5708   -0.2292   -0.0215   -0.2569    0.2692    0.3185   -0.7785    0.5854   -0.8162   -0.3246   -0.1754    0.5677    0.2908


sort_coord =

   -0.8162   -0.7785   -0.3246   -0.2569   -0.2292   -0.1754   -0.0215    0.2692    0.2908    0.3185    0.5677    0.5708    0.5854


permut =

     9
     7
    10
     4
     2
    11
     3
     5
    13
     6
    12
     1
     8


prox_permut =

         0    0.1600    0.8200    0.9400    0.9000    0.9400    0.9600    0.9600    1.0000    0.9800    0.9800    1.0000    0.9700
   -0.1600         0    0.7200    0.9800    0.8600    0.9200    0.9600    0.9600    0.9800    1.0000    1.0000    0.9800    0.9200
   -0.8200   -0.7200         0    0.2600    0.1800    0.1600    0.2200    0.7600    0.8400    0.8200    0.8000    0.8800    0.8400
   -0.9400   -0.9800   -0.2600         0   -0.0400    0.2400    0.5000    0.6200    0.7400    0.9000    0.8200    0.9000    0.8400
   -0.9000   -0.8600   -0.1800    0.0400         0   -0.0200    0.4200    0.5200    0.6200    0.8400    0.8000    0.8600    0.8400
   -0.9400   -0.9200   -0.1600   -0.2400    0.0200         0    0.1600    0.5600    0.5600    0.4800    0.9600    0.9600    0.8400
   -0.9600   -0.9600   -0.2200   -0.5000   -0.4200   -0.1600         0    0.3400    0.3600    0.5000    0.7400    0.8400    0.7200
   -0.9600   -0.9600   -0.7600   -0.6200   -0.5200   -0.5600   -0.3400         0    0.1000   -0.0200    0.2800    0.4600    0.4000
   -1.0000   -0.9800   -0.8400   -0.7400   -0.6200   -0.5600   -0.3600   -0.1000         0         0    0.4600    0.5800    0.3800
   -0.9800   -1.0000   -0.8200   -0.9000   -0.8400   -0.4800   -0.5000    0.0200         0         0    0.3600    0.4200    0.5800
   -0.9800   -1.0000   -0.8000   -0.8200   -0.8000   -0.9600   -0.7400   -0.2800   -0.4600   -0.3600         0   -0.4600    0.2800
   -1.0000   -0.9800   -0.8800   -0.9000   -0.8600   -0.9600   -0.8400   -0.4600   -0.5800   -0.4200    0.4600         0         0
   -0.9700   -0.9200   -0.8400   -0.8400   -0.8400   -0.8400   -0.7200   -0.4000   -0.3800   -0.5800   -0.2800         0         0


vaf =

    0.9126


alpha_multiplier =

    0.1205


alpha_vaf =

    0.8690


alpha_coord =

   -0.7231
   -0.6026
   -0.4821
   -0.3616
   -0.2410
   -0.1205
         0
    0.1205
    0.2410
    0.3616
    0.4821
    0.6026
    0.7231

>> [coord,sort_coord,permut,prox_permut,vaf,alpha_multiplier,alpha_vaf,...
alpha_coord] = skew_symmetric_scaling(thurstone_skew_symmetric_after)

coord =

    0.5446    0.0200   -0.0492   -0.2569    0.2815    0.2677   -0.7446    0.5508   -0.8138   -0.3908   -0.2092    0.5138    0.2862


sort_coord =

   -0.8138   -0.7446   -0.3908   -0.2569   -0.2092   -0.0492    0.0200    0.2677    0.2815    0.2862    0.5138    0.5446    0.5508


permut =

     9
     7
    10
     4
    11
     3
     2
     6
     5
    13
    12
     1
     8


prox_permut =

         0    0.2800    0.7400    0.9200    0.9000    0.9600    0.9200    0.9600    0.9800    0.9600    0.9800    1.0000    0.9800
   -0.2800         0    0.5800    0.8800    0.8800    0.9400    0.9000    0.9600    0.9600    0.9600    0.9600    0.9600    0.9800
   -0.7400   -0.5800         0    0.3400    0.2800    0.3600    0.4600    0.8000    0.7800    0.7800    0.8400    0.8800    0.8800
   -0.9200   -0.8800   -0.3400         0    0.0600    0.4000    0.4000    0.7400    0.6800    0.6800    0.8400    0.9000    0.7800
   -0.9000   -0.8800   -0.2800   -0.0600         0    0.2400    0.2800    0.5200    0.6200    0.5200    0.9400    0.9400    0.7800
   -0.9600   -0.9400   -0.3600   -0.4000   -0.2400         0    0.0200    0.4000    0.4400    0.4000    0.6800    0.8600    0.7400
   -0.9200   -0.9000   -0.4600   -0.4000   -0.2800   -0.0200         0    0.3800    0.2800    0.3800    0.4600    0.5800    0.6400
   -0.9600   -0.9600   -0.8000   -0.7400   -0.5200   -0.4000   -0.3800         0         0    0.0200    0.4000    0.4000    0.4600
   -0.9800   -0.9600   -0.7800   -0.6800   -0.6200   -0.4400   -0.2800         0         0    0.0800    0.3000    0.3400    0.3600
   -0.9600   -0.9600   -0.7800   -0.6800   -0.5200   -0.4000   -0.3800   -0.0200   -0.0800         0    0.2800    0.4600    0.3200
   -0.9800   -0.9600   -0.8400   -0.8400   -0.9400   -0.6800   -0.4600   -0.4000   -0.3000   -0.2800         0   -0.2400    0.2400
   -1.0000   -0.9600   -0.8800   -0.9000   -0.9400   -0.8600   -0.5800   -0.4000   -0.3400   -0.4600    0.2400         0         0
   -0.9800   -0.9800   -0.8800   -0.7800   -0.7800   -0.7400   -0.6400   -0.4600   -0.3600   -0.3200   -0.2400         0         0


vaf =

    0.9340


alpha_multiplier =

    0.1164


alpha_vaf =

    0.8927


alpha_coord =

   -0.6982
   -0.5819
   -0.4655
   -0.3491
   -0.2327
   -0.1164
         0
    0.1164
    0.2327
    0.3491
    0.4655
    0.5819
    0.6982
  \end{verbatim}

  \large

  \smallskip

  Given a skew-symmetric matrix $\{p_{ij}^{ss}\}$ (or possibly, just $\{\mathrm{sign}(p_{ij}^{ss}\}$), an obvious class of measures for matrix pattern based on matrix reordering would be the sum of the above-diagonal entries:
  \[ \sum_{i < j} p_{ij}^{ss}  (\mathrm{or} \sum_{i < j} \mathrm{sign}(p_{ij}^{ss})) \ . \]
  \noindent In fact, because $p_{ij}^{ss} = \mathrm{sign}|p_{ij}^{ss}|$, the index $\sum_{i < j} p_{ij}^{ss}$ can be interpreted merely as a weighted version of the one based just on $\mathrm{sign}(p_{ij}^{ss})$.  We will assume in our discussion below that  $\sum_{i < j} p_{ij}^{ss}$ is being considered, but the obvious replacement of $p_{ij}^{ss}$ by $\mathrm{sign}(p_{ij}^{ss})$ could incorporate the use of $\sum_{i < j} \mathrm{sign}(p_{ij}^{ss})$ directly.

  \smallskip

  A dynamic programming solution to the optimization task of reordering the rows/columns of a matrix to maximize the sum of above-diagonal entries, was first sketched by Lawler (1964) in identifying minimum feedback arc sets in a directed graph.  The M-file that we use below on the Thurstone skew-symmetric matrices,  \verb+lawler_dp.m+, implements the dynamic programming recursion.
    The optimization task itself, however, has several other distinct substantive incarnations, e.g., in maximum likelihood paired comparison ranking (Flueck and Korsh, 1974), or to triangulating an input-output matrix (Korte and Oberhofer, 1971).  For an extensive review of the variety of possible applications up through the middle 1970's, the reader is referred to Hubert (1976).

    \smallskip

    The syntax of \verb+skew_symmetric_lawler_dp.m+ is as follows:
    \begin{verbatim}
    [permut,prox_permut,cumobfun] = skew_symmetric_lawler_dp(prox)
    \end{verbatim}
    \noindent Here, \verb+PROX+ is the input skew-symmetric proximity matrix (with a zero main
   diagonal);
   \verb+PERMUT+ is the order of the objects in the optimal permutation;
   \verb+PROX_PERMUT+ is the reordered proximity matrix using \verb+PERMUT+;
   \verb+CUMOBFUN+ gives the cumulative values of the objective function for
   the successive placements of the objects in the optimal permutation.  The verbatim output below is an implementation of \verb+lawler_dp.m+ on the two Thurstone `before' and `after' skew-symmetric proximity matrices.  The offense of `gambler' (number 2) moves appropriately after the movie has been shown.

   \tiny

   \begin{verbatim}
   >> [permut,prox_permut,cumobfun] = skew_symmetric_lawler_dp(thurstone_skew_symmetric_before)
Elapsed time is 0.258336 seconds.

permut =

     9
     7
    10
     2
     4
    11
     3
     6
     5
    13
     1
    12
     8


prox_permut =

         0    0.1600    0.8200    0.9000    0.9400    0.9400    0.9600    0.9800    0.9600    1.0000    1.0000    0.9800    0.9700
   -0.1600         0    0.7200    0.8600    0.9800    0.9200    0.9600    1.0000    0.9600    0.9800    0.9800    1.0000    0.9200
   -0.8200   -0.7200         0    0.1800    0.2600    0.1600    0.2200    0.8200    0.7600    0.8400    0.8800    0.8000    0.8400
   -0.9000   -0.8600   -0.1800         0    0.0400   -0.0200    0.4200    0.8400    0.5200    0.6200    0.8600    0.8000    0.8400
   -0.9400   -0.9800   -0.2600   -0.0400         0    0.2400    0.5000    0.9000    0.6200    0.7400    0.9000    0.8200    0.8400
   -0.9400   -0.9200   -0.1600    0.0200   -0.2400         0    0.1600    0.4800    0.5600    0.5600    0.9600    0.9600    0.8400
   -0.9600   -0.9600   -0.2200   -0.4200   -0.5000   -0.1600         0    0.5000    0.3400    0.3600    0.8400    0.7400    0.7200
   -0.9800   -1.0000   -0.8200   -0.8400   -0.9000   -0.4800   -0.5000         0    0.0200         0    0.4200    0.3600    0.5800
   -0.9600   -0.9600   -0.7600   -0.5200   -0.6200   -0.5600   -0.3400   -0.0200         0    0.1000    0.4600    0.2800    0.4000
   -1.0000   -0.9800   -0.8400   -0.6200   -0.7400   -0.5600   -0.3600         0   -0.1000         0    0.5800    0.4600    0.3800
   -1.0000   -0.9800   -0.8800   -0.8600   -0.9000   -0.9600   -0.8400   -0.4200   -0.4600   -0.5800         0    0.4600         0
   -0.9800   -1.0000   -0.8000   -0.8000   -0.8200   -0.9600   -0.7400   -0.3600   -0.2800   -0.4600   -0.4600         0    0.2800
   -0.9700   -0.9200   -0.8400   -0.8400   -0.8400   -0.8400   -0.7200   -0.5800   -0.4000   -0.3800         0   -0.2800         0


cumobfun =

         0
    0.1600
    1.7000
    3.6400
    5.8600
    8.1000
   11.3200
   16.8400
   21.5800
   26.7800
   34.6600
   42.3200
   49.9300

>> [permut,prox_permut,cumobfun] = skew_symmetric_lawler_dp(thurstone_skew_symmetric_after)
Elapsed time is 0.234843 seconds.

permut =

     9
     7
    10
     4
    11
     3
     2
     5
     6
    13
     1
    12
     8


prox_permut =

         0    0.2800    0.7400    0.9200    0.9000    0.9600    0.9200    0.9800    0.9600    0.9600    1.0000    0.9800    0.9800
   -0.2800         0    0.5800    0.8800    0.8800    0.9400    0.9000    0.9600    0.9600    0.9600    0.9600    0.9600    0.9800
   -0.7400   -0.5800         0    0.3400    0.2800    0.3600    0.4600    0.7800    0.8000    0.7800    0.8800    0.8400    0.8800
   -0.9200   -0.8800   -0.3400         0    0.0600    0.4000    0.4000    0.6800    0.7400    0.6800    0.9000    0.8400    0.7800
   -0.9000   -0.8800   -0.2800   -0.0600         0    0.2400    0.2800    0.6200    0.5200    0.5200    0.9400    0.9400    0.7800
   -0.9600   -0.9400   -0.3600   -0.4000   -0.2400         0    0.0200    0.4400    0.4000    0.4000    0.8600    0.6800    0.7400
   -0.9200   -0.9000   -0.4600   -0.4000   -0.2800   -0.0200         0    0.2800    0.3800    0.3800    0.5800    0.4600    0.6400
   -0.9800   -0.9600   -0.7800   -0.6800   -0.6200   -0.4400   -0.2800         0         0    0.0800    0.3400    0.3000    0.3600
   -0.9600   -0.9600   -0.8000   -0.7400   -0.5200   -0.4000   -0.3800         0         0    0.0200    0.4000    0.4000    0.4600
   -0.9600   -0.9600   -0.7800   -0.6800   -0.5200   -0.4000   -0.3800   -0.0800   -0.0200         0    0.4600    0.2800    0.3200
   -1.0000   -0.9600   -0.8800   -0.9000   -0.9400   -0.8600   -0.5800   -0.3400   -0.4000   -0.4600         0    0.2400         0
   -0.9800   -0.9600   -0.8400   -0.8400   -0.9400   -0.6800   -0.4600   -0.3000   -0.4000   -0.2800   -0.2400         0    0.2400
   -0.9800   -0.9800   -0.8800   -0.7800   -0.7800   -0.7400   -0.6400   -0.3600   -0.4600   -0.3200         0   -0.2400         0


cumobfun =

         0
    0.2800
    1.6000
    3.7400
    5.8600
    8.7600
   11.7400
   16.4800
   21.2400
   26.0200
   33.3400
   40.2600
   47.4200
   \end{verbatim}

   \large

\section{An Alternative Rooting Strategy for Additive Trees}

The earlier Cluster Analysis Toolbox described (in Section 6), the issue of working with additive trees and finding sensible spots to root a tree.  One strategy, defined by the M-file, \verb+atreedec.m+, decomposed any given additive tree (matrix) into an ultrametric and a centroid metric (matrix), where the root is situated halfway along the longest path.  That particular strategy can be generalized so a root can be placed half-way along the path containing \emph{any} two of the terminal nodes.  An M-file, called \verb+atreefnd_root.m+, is available that has the same syntax as \verb+atreefnd.m+, but in the process of running, produces the information necessary to root the tree at any of the $n(n-1)/2$ path midpoints.  An example is given below on \verb+number.dat+, where the information for tooting halfway between the objects 1 and 2, and halfway between 5 and 9, is given (i.e., the information for the other pairs is deleted from the verbatim output below).  Figures 4 and 5 give the visually different representations, where the toots are indicated by the larger filled-in circle. (Remember, the labeling of the objects from 1 to 10 in the Figures and the verbatim output, correspond to the digits from 0 to 9, respectively.)

\begin{figure}

\begin{center}

\caption{A Graph-Theoretic Representation for the Additive Tree Obtained for the Digit Data; the Root is Midway on the Path Between the Objects Labeled 1 and 2 (the digits 0 and 1)}

\setlength{\unitlength}{.5pt}

\begin{picture}(500,1000)(0,-250)

\put(50,263){\circle{20}}

\put(100,-34){\circle{20}}

\put(150,-25){\circle{20}}

\put(200,-71){\circle{20}}

\put(250,-135){\circle{20}}

\put(300,-183){\circle{20}}

\put(350,-27){\circle{20}}

\put(400,-148){\circle{20}}

\put(450,-53){\circle{20}}

\put(500,263){\circle{20}}

\put(125,0){\circle*{20}}

\put(162.5,103){\circle*{20}}

\put(275,41){\circle*{20}}

\put(331.22,123){\circle*{20}}

\put(387.5,85){\circle*{20}}

\put(425,31){\circle*{20}}

\put(246.86,201){\circle*{20}}

\put(373.38,381){\circle*{20}}

\put(211.69,474){\circle*{30}}

\put(50,213){\makebox(0,0){1}}

\put(100,-84){\makebox(0,0){3}}

\put(150,-75){\makebox(0,0){5}}

\put(200,-121){\makebox(0,0){9}}

\put(250,-185){\makebox(0,0){6}}

\put(300,-233){\makebox(0,0){8}}

\put(350,-77){\makebox(0,0){7}}

\put(400,-198){\makebox(0,0){10}}

\put(450,-103){\makebox(0,0){4}}

\put(500,213){\makebox(0,0){2}}

\put(50,273){\line(0,1){201}}

\put(100,-24){\line(0,1){24}}

\put(150,-15){\line(0,1){15}}

\put(200,-61){\line(0,1){164}}

\put(250,-125){\line(0,1){166}}

\put(300,-173){\line(0,1){214}}

\put(350,-17){\line(0,1){102}}

\put(400,-138){\line(0,1){169}}

\put(450,-43){\line(0,1){74}}

\put(500,273){\line(0,1){108}}

\put(125,0){\line(0,1){103}}

\put(162.5,103){\line(0,1){98}}

\put(275,41){\line(0,1){82}}

\put(331.22,123){\line(0,1){78}}

\put(387.5,85){\line(0,1){38}}

\put(425,31){\line(0,1){54}}

\put(246.86,201){\line(0,1){180}}

\put(373.38,381){\line(0,1){93}}

\put(211.69,474){\line(0,1){50}}

\put(100,0){\line(1,0){50}}

\put(250,41){\line(1,0){50}}

\put(350,85){\line(1,0){75}}

\put(400,31){\line(1,0){50}}

\put(125,103){\line(1,0){75}}

\put(275,123){\line(1,0){112.85}}

\put(162.5,201){\line(1,0){168.72}}

\put(246.86,381){\line(1,0){253.14}}

\put(50,474){\line(1,0){323.38}}








\end{picture}

\end{center}

\end{figure}


\begin{figure}

\begin{center}

\caption{A Graph-Theoretic Representation for the Additive Tree Obtained for the Digit Data; the Root is Midway on the Path Between the Objects Labeled 5 and 9 (the digits 4 and 8)}

\setlength{\unitlength}{.5pt}

\begin{picture}(500,1000)(0,-250)

\put(50,-203){\circle{20}}

\put(100,-155){\circle{20}}

\put(150,-47){\circle{20}}

\put(200,-168){\circle{20}}

\put(250,-73){\circle{20}}

\put(300,-303){\circle{20}}

\put(350,-118){\circle{20}}

\put(400,141){\circle{20}}

\put(450,150){\circle{20}}

\put(500,150){\circle{20}}

\put(75,21){\circle*{20}}

\put(125,103){\circle*{20}}

\put(175,65){\circle*{20}}

\put(225,11){\circle*{20}}

\put(225,181){\circle*{20}}

\put(325,278){\circle*{20}}

\put(325,0){\circle*{20}}

\put(425,175){\circle*{20}}

\put(462.5,300){\circle*{30}}

\put(50,-253){\makebox(0,0){8}}

\put(100,-205){\makebox(0,0){6}}

\put(150,-97){\makebox(0,0){7}}

\put(200,-218){\makebox(0,0){10}}

\put(250,-123){\makebox(0,0){4}}

\put(300,-353){\makebox(0,0){1}}

\put(350,-168){\makebox(0,0){2}}

\put(400,91){\makebox(0,0){3}}

\put(450,100){\makebox(0,0){5}}

\put(500,100){\makebox(0,0){9}}



\put(50,-193){\line(0,1){214}}

\put(100,-145){\line(0,1){166}}

\put(150,-37){\line(0,1){102}}

\put(200,-158){\line(0,1){169}}

\put(250,-63){\line(0,1){74}}

\put(300,-293){\line(0,1){293}}

\put(350,-108){\line(0,1){108}}

\put(400,151){\line(0,1){24}}

\put(450,160){\line(0,1){15}}

\put(500,160){\line(0,1){140}}

\put(75,21){\line(0,1){82}}

\put(125,103){\line(0,1){78}}

\put(175,65){\line(0,1){38}}

\put(225,11){\line(0,1){54}}

\put(225,181){\line(0,1){97}}

\put(325,278){\line(0,1){22}}

\put(325,0){\line(0,1){181}}

\put(425,175){\line(0,1){103}}

\put(462.5,300){\line(0,1){50}}


\put(50,21){\line(1,0){50}}

\put(75,103){\line(1,0){100}}

\put(150,65){\line(1,0){75}}

\put(125,181){\line(1,0){200}}

\put(200,11){\line(1,0){50}}

\put(225,278){\line(1,0){200}}

\put(300,0){\line(1,0){50}}

\put(325,300){\line(1,0){175}}

\put(400,175){\line(1,0){50}}








\end{picture}

\end{center}

\end{figure}


\bigskip

\tiny

\begin{verbatim}
>> load number.dat
>> [find,vaf] = atreefnd_root(number,randperm(10))

pairindexes =

     1     2


orderprox =

         0    0.9477    0.9477    0.9477    0.9477    0.9477    0.9477    0.9477    0.9477    0.9477
    0.9477         0         0    0.2064    0.4014    0.4014    0.4014    0.4014    0.4014    0.7626
    0.9477         0         0    0.2064    0.4014    0.4014    0.4014    0.4014    0.4014    0.7626
    0.9477    0.2064    0.2064         0    0.4014    0.4014    0.4014    0.4014    0.4014    0.7626
    0.9477    0.4014    0.4014    0.4014         0    0.0829    0.2465    0.2465    0.2465    0.7626
    0.9477    0.4014    0.4014    0.4014    0.0829         0    0.2465    0.2465    0.2465    0.7626
    0.9477    0.4014    0.4014    0.4014    0.2465    0.2465         0    0.1707    0.1707    0.7626
    0.9477    0.4014    0.4014    0.4014    0.2465    0.2465    0.1707         0    0.0628    0.7626
    0.9477    0.4014    0.4014    0.4014    0.2465    0.2465    0.1707    0.0628         0    0.7626
    0.9477    0.7626    0.7626    0.7626    0.7626    0.7626    0.7626    0.7626    0.7626         0


orderperm =

     1     3     5     9     6     8     7    10     4     2


centroid_metric =

   -0.2634    0.0341    0.0249    0.0706    0.1345    0.1826    0.0272    0.1475    0.0527   -0.2634

-----------------------

pairindexes =

     5     9

Elapsed time is 0.001798 seconds.
Elapsed time is 0.015806 seconds.

orderprox =

         0    0.0428    0.2064    0.2064    0.2064    0.3612    0.3612    0.5562    0.5562    0.6019
    0.0428         0    0.2064    0.2064    0.2064    0.3612    0.3612    0.5562    0.5562    0.6019
    0.2064    0.2064         0    0.1305    0.1305    0.3612    0.3612    0.5562    0.5562    0.6019
    0.2064    0.2064    0.1305         0    0.0227    0.3612    0.3612    0.5562    0.5562    0.6019
    0.2064    0.2064    0.1305    0.0227         0    0.3612    0.3612    0.5562    0.5562    0.6019
    0.3612    0.3612    0.3612    0.3612    0.3612         0         0    0.5562    0.5562    0.6019
    0.3612    0.3612    0.3612    0.3612    0.3612         0         0    0.5562    0.5562    0.6019
    0.5562    0.5562    0.5562    0.5562    0.5562    0.5562    0.5562         0    0.3498    0.6019
    0.5562    0.5562    0.5562    0.5562    0.5562    0.5562    0.5562    0.3498         0    0.6019
    0.6019    0.6019    0.6019    0.6019    0.6019    0.6019    0.6019    0.6019    0.6019         0


orderperm =

     8     6     7    10     4     1     2     3     5     9


centroid_metric =

    0.2027    0.1545    0.0473    0.1675    0.0728    0.3031    0.1179   -0.1408   -0.1500   -0.1500
\end{verbatim}

\large

\newpage

\begin{thebibliography}{99}



\bibitem{} Bodewig, E. (1956). \emph{Matrix calculus}. Amsterdam: North-Holland.



\bibitem{} Carroll, J. D. (1992).  Metric, nonmetric, and quasi-nonmetric analysis of psychological data.  Division 5 Presidential Address, American Psychological Association, Washington, DC, August, 1992 (published in \emph{Score}, Newsletter of Division 5, October, 1992, pp. 4--5).






\bibitem{} Coombs, C. H. (1964). \emph{A theory of data}. New York: Wiley.


\bibitem{} Critchley, F. (1994). On exchangeability-based equivalence
relations induced by strongly Robinson and, in particular, by
quadripolar Robinson dissimilarity matrices.  In B. van Cutsem
(Ed.), \emph{Classification and dissimilarity analysis}, Lecture
Notes in Statistics (pp.\ 173--199).  New York: Springer-Verlag.

\bibitem{} Critchley, F., \& Fichet, B. (1994). The partial order by
inclusion of the principal classes of dissimilarity on a finite
set, and some of their basic properties. In B. van Cutsem (Ed.),
\emph{Classification and dissimilarity analysis}, Lecture Notes
in Statistics (pp.\ 5--65).  New York: Springer-Verlag.




\bibitem{} Durand, C., \& Fichet, B. (1988). One-to-one correspondences in
pyramidal representations: A unified approach.  In H. H. Bock
(Ed.), \emph{Classification and related methods of data analysis}
(pp.\ 85--90). Amsterdam: North-Holland.


\bibitem{} Dykstra, R. L. (1983). An algorithm for restricted least squares regression.  \emph{Journal of the American Statistical Association}, \emph{78}, 837--842.

\bibitem{} Flueck, J. A., \& Korsh, J. F. (1974).  A branch search
algorithm for maximum likehihood paired comparison ranking.
\emph{Biometrika}, \emph{61}, 621--626.



\bibitem{} Hubert, L. J. (1974).  Problems of seriation using a subject by item response matrix.  \emph{Psychological Bulletin}, \emph{81}, 976--983.


\bibitem{} Hubert, L. J. (1976). Seriation using asymmetric
proximity measures. \emph{British Journal of Mathematical and
Statistical Psychology}, \emph{29}, 32--52.


\bibitem{} Hubert, L. J., \& Arabie, P. (1994). The analysis of proximity
matrices through sums of matrices having (anti-)Robinson forms.
\emph{British Journal of Mathematical and Statistical
Psychology}, \emph{47}, 1--40.

\bibitem{} Hubert, L. J., \& Arabie, P. (1995). The approximation of two-mode proximity matrices by sums of order-constrained matrices. \emph{Psychometrika}, \emph{60}, 573--605.






\bibitem{} Hubert, L. J., Arabie, P., \& Meulman, J. J. (1997). Linear and
circular unidimensional scaling for symmetric proximity matrices.
\emph{British Journal of Mathematical and Statistical
Psychology}, \emph{50}, 253--284.



 \bibitem{} Hubert, L. J., Arabie, P., \& Meulman, J. J. (1998)  Graph-theoretic representations for proximity matrices through strongly-anti-Robinson or circular strongly-anti-Robinson matrices. \emph{Psychometrika}, \emph{63}, 341--358.

\bibitem{} Hubert, L. J., Arabie, P., \& Meulman, J. J. (2001). \emph{Combinatorial data analysis: Optimization by dynamic programming}. Philadelphia: SIAM.


\bibitem{} Hubert, L. J., Arabie, P., \& Meulman, J. J. (2006). \emph{The structural representation of proximity matrices with MATLAB}. Philadelphia: SIAM.






\bibitem{} Kaczmarz, S. (1937). Angen\"{a}herte Aufl\"{o}sung von Systemen linearer Gleichungen. \emph{Bulletin of the Polish Academy of Sciences}, \emph{A35}, 355--357.


\bibitem{} Korte, B., \& Oberhofer, W. (1971). Triangularizing
input-output matrices and the structure of production.
\emph{European Economic Review}, \emph{2}, 493--522.


\bibitem{} Kruskal, J. B. (1964a).  Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis.  \emph{Psychometrika}, \emph{29}, 1--27.




 \bibitem{} Kruskal, J. B. (1964b).  Nonmetric multidimensional scaling: A numerical method.  \emph{Psychometrika},
 \emph{29}, 115--129.




\bibitem{} Kruskal, J. B., Young, F. W., \& Seery, J. B. (1977). \emph{How to use KYST2, a very flexible program to do multidimensional  scaling and unfolding}. AT\&T Bell Laboratories, Murray Hill, NJ.



\bibitem{} Lawler, E. L. (1964). A comment on minimum feedback arc
sets. \emph{IEEE Tranctions of Circuit Theory}, \emph{11}, 296--297.



\bibitem{} Mirkin, B. (1996). \emph{Mathematical classification and clustering}.  Dordrecht: Kluwer.





\bibitem{} Plutchik, R. \& Conte, H. R. (Eds.). (1997). \emph{Circumplex models of personality and emotions}. Washington, DC: American Psychological Association.


\bibitem{} Ramsay, J. O. (1988).  Monotone regression splines in action.  \emph{Statistical Science}, \emph{3}, 425--461.


\bibitem{} Shepard, R. N. (1974). Representation of structure in similarity data: Problems and prospects. \emph{Psychometrika}, \emph{39}, 373--421.


\bibitem{} Shepard, R. N., Kilpatric, D. W., \& Cunningham, J. P. (1975).
The internal representation of numbers. \emph{Cognitive
Psychology}, \emph{7}, 82--138.

\bibitem{} Thurstone, L. L. (1959). \emph{The measurement of
values}. Chicago: University of Chicago Press.


\bibitem{} Wilkinson, L. (1988).  \emph{SYSTAT: The System for Statistics}.   SYSTAT, Inc, Evanston, IL.

\bibitem{} Wollan, P. C., \& Dykstra, R. L. (1987).  Minimizing linear constrained Mahalanob is distances.  \emph{Applied Statistics}, \emph{36}, 234--240.

\end{thebibliography}


\newpage



\appendix

\section{Header Comments for the M-files Mentioned in the Text or Used Internally by Other M-files;
Given in Alphabetical Order}

\small

\section*{}

\end{document}
