\documentclass[notitlepage,12pt]{article}%
\usepackage{amssymb}
\usepackage{graphicx}
\usepackage{amsmath}
\usepackage{amsfonts}%
\setcounter{MaxMatrixCols}{30}
%TCIDATA{OutputFilter=latex2.dll}
%TCIDATA{Version=5.00.0.2606}
%TCIDATA{CSTFile=LaTeX article.cst}
%TCIDATA{Created=Fri Dec 24 08:35:56 1999}
%TCIDATA{LastRevised=Tuesday, August 23, 2005 08:29:50}
%TCIDATA{<META NAME="GraphicsSave" CONTENT="32">}
%TCIDATA{<META NAME="SaveForMode" CONTENT="1">}
%TCIDATA{BibliographyScheme=Manual}
%TCIDATA{<META NAME="DocumentShell" CONTENT="Journal Articles\Standard LaTeX Article">}
%TCIDATA{Language=American English}
\input jslfloat
\input jsl1in
\raggedbottom
\begin{document}

\begin{center}
{\Large Confidence Intervals for Predicted Outcomes in Regression Models for
Categorical Outcomes}{\footnotesize \bigskip}

{\large Jun Xu and J.\ Scott Long}\bigskip

Indiana University \bigskip

August 23, 2005{\large \bigskip}
\end{center}

\section{Overview}

The interpretation of regression models often involves the examination of
predicted outcomes at specific values of the independent variables. The
\texttt{prvalue} command by Long and Freese (forthcoming) makes it very simple
to compute such post-estimation predictions. After estimating a model, the
user specifies values of interest for the independent variables and
\texttt{prvalue} computes the predicted value (or values) of the outcome.
Changes in the predictions as values of the independent variables change can
also be computed. In this article, we describe enhancements to
\texttt{prvalue} that allow the computation of confidence intervals for both
predicted outcomes and discrete changes in predictions. Depending on the
regression model and options chosen, confidence intervals are computed using
standard maximum likelihood methods, the method of endpoint transformations,
the delta method, or bootstrapping. The command works after estimating models
using \texttt{cloglog}, \texttt{cnreg}, \texttt{intreg}, \texttt{logit},
\texttt{mlogit}, \texttt{nbreg}, \texttt{ologit}, \texttt{oprobit},
\texttt{poisson}, \texttt{probit}, \texttt{regress}, \texttt{tobit},
\texttt{zinb}, and \texttt{zip}. To complement \texttt{prvalue}, we also
present enhancements to \texttt{prgen} that allow you to plot confidence
intervals and marginal changes.

\section{Predictions and Methods for Computing Confidence Intervals}

For a variety of regression models, \texttt{prvalue} computes predicted
outcomes along with confidence intervals for these predictions. For models
with categorical outcomes, the probability of each outcome is computed. For
example, with the binary probit model the probability of observing a one is
estimated as $\widehat{\pi}=\Phi\left(  \mathbf{x}^{\prime}\widehat
{\boldsymbol{\beta}}\right)  $, where $\mathbf{x}$ is a vector of independent
variables and $\widehat{\boldsymbol{\beta}}$ contain the estimated parameters.
We can also estimate a confidence interval for $\pi$. For a given confidence
level, the upper and lower bounds define the confidence interval. For example,
the 95\% confidence interval for $\pi$ includes upper and lower bounds such
that
\[
\Pr\left(  \pi_{\text{LB}}\leq\pi\leq\pi_{\text{UB}}\right)  =.95\ .
\]
To understand what this means, imagine that we take repeated samples from our
population and that for each sample we estimate the upper and lower bounds of
the confidence interval for that probability. About 95\% of these confidence
intervals would contain the true probability $\pi$.

For count models, both the predicted rate and the probability of each count
are computed along with confidence intervals. For example, with the Poisson
regression model the expected rate is $\widehat{\mu}=\exp\left(
\mathbf{x}\widehat{\boldsymbol{\beta}}\right)  $ and we compute the confidence
interval such that $\Pr\left(  \mu_{\text{LB}}\leq\mu\leq\mu_{\text{UB}%
}\right)  =.95.$With count models, we also compute probabilities of specific
values of the outcome. For example, we might compute the probability
$\Pr\left(  y=2\mid\mathbf{x}\right)  $ of observing a count of two given the
levels of the independent variables, with the confidence interval such that%
\[
\Pr\left[  \Pr\left(  y=2\mid\mathbf{x}\right)  _{\text{LB}}\leq\Pr\left(
y=2\mid\mathbf{x}\right)  \leq\Pr\left(  y=2\mid\mathbf{x}\right)
_{\text{UB}}\right]  =.95\ .
\]
The values computed for each model are summarized here:\footnote{With Stata 9,
\texttt{prvalue} can also compute predicted outcomes for the models estimated
by \texttt{mprobit}, \texttt{slogit}, \texttt{ztp} and \texttt{ztnb}. However,
confidence intervals are not available for these commands.}\medskip\medskip%

\begin{tabular}
[c]{lcccc}\hline\hline
Models{\LARGE \ \ \ } & $\Pr\left(  y\right)  $ & $y$ or $y^{\ast}$ & Rate
$\mu$ & $\Pr\left(  \text{count}\right)  $\\\hline
\texttt{cloglog}, \texttt{logit}, \texttt{probit} & default & \texttt{ystar}%
$^{1}$ & no & no\\
\texttt{ologit}, \texttt{oprobit} & default & \texttt{ystar}$^{1}$ & no & no\\
\texttt{mlogit}, & default & no & no & no\\
\texttt{zinb}, \texttt{zip}, \texttt{nbreg}, \texttt{poisson} & no & no &
default & default\\
\texttt{regress}, \texttt{cnreg}, \texttt{intreg}, \texttt{tobit} & no &
default & no & no\\\hline\hline
{\small 1: when the \texttt{ystar} option is specified}. &
\multicolumn{1}{l}{} & \multicolumn{1}{l}{} & \multicolumn{1}{l}{} &
\multicolumn{1}{l}{}%
\end{tabular}
\medskip\medskip

We can also compute changes in the predicted outcome as levels of the $x$'s
change. For example, suppose that we have two sets of values for the
independent variables, $\mathbf{x}_{A}$ and $\mathbf{x}_{B}$. We could compute
how the predicted outcome changes when the values of $\mathbf{x}$ change from
$\mathbf{x}_{A}$ and $\mathbf{x}_{B}$. For example, for a binary model this
would be%
\[
\frac{\Delta\Pr\left(  y=1\right)  }{\Delta\mathbf{x}}=\Pr\left(
y=1\mid\mathbf{x}_{A}\right)  -\Pr\left(  y=1\mid\mathbf{x}_{B}\right)  \text{
,}%
\]
along with the confidence interval%
\[
\Pr\left[  \frac{\Delta\Pr\left(  y=1\right)  }{\Delta\mathbf{x}}_{\text{LB}%
}\leq\frac{\Delta\Pr\left(  y=1\right)  }{\Delta\mathbf{x}}\leq\frac{\Delta
\Pr\left(  y=1\right)  }{\Delta\mathbf{x}}_{\text{UB}}\right]  =.95\ .
\]
For further details, see Long and Freese (forthcoming).

\texttt{prvalue} uses four methods to compute confidence intervals: 1) maximum
likelihood, 2) endpoint transformation, 3) the bootstrap method, and 4) the
delta method. While the technical details on each method for are given in
Section \ref{CISection}, here we provide general information about each method.

\begin{enumerate}
\item \textbf{Maximum Likelihood} For models such as the linear regression
model, the standard method of computing confidence intervals using maximum
likelihood theory is used.

\item \textbf{Endpoint Transformation} For binary models, this method computes
the upper and lower bounds of the confidence intervals for the linear
combination $\mathbf{x}^{\prime}\boldsymbol{\beta}$ and then transforms these
bounds into the probability of observing a one. For example for probit, the
lower bound $\left(  \mathbf{x}^{\prime}\boldsymbol{\beta}\right)
_{\text{LB}}$ would be transformed into the lower bound for the probability as
$\Pr\left(  y=1\right)  _{\text{LB}}=\Phi\left[  \left(  \mathbf{x}^{\prime
}\boldsymbol{\beta}\right)  _{\text{LB}}\right]  $. One advantage of this
method is that the bounds cannot be smaller than 0 or greater than 1. This
method cannot be used for computing confidence intervals for changes in predictions.

\item \textbf{Delta Method} This method takes a function that is too complex
for analytically computing the variance (for example, the change in the
predicted probabilities in a multinomial logit model) and creates a linear
approximation of that function. The variance of the simpler approximation is
used for constructing the confidence interval. Since \texttt{prvalue} uses
analytic formulas for the derivatives, rather than numerical estimation, the
computation of confidence intervals is extremely fast. Unlike the method of
endpoint transformation, the bounds computed by the delta method can include
values that exceed the range of the statistic being estimated (e.g., a bound
for a predicted probability could be negative or greater than one).

\item \textbf{Bootstrap Method} The idea of the bootstrap (see Guan 2003 for
an introduction to the bootstrap using Stata) is that by taking repeated
samples from the sample used to estimate your model, you can estimate the
sampling variability that would occur by taking repeated samples from the
population. This is done by taking a random sample from the estimation sample,
computing the statistics of interest, and repeating this for some number of
replications. The variation in the estimates across the replications is used
to estimate the standard deviation of the sampling distribution. While the
bootstrap method generally works quite well and avoids assumptions implicit in
other methods, it is computationally intensive. For example, computing the
confidence intervals for a multinomial logit with 5 outcomes, three variables,
and 337 cases by the delta method took .15 seconds, while computing the
confidence intervals by bootstrap took 141 seconds using 1,000 replications.
Computations of confidence intervals for a multinomial logit with 6 outcomes,
4 variables and 7,357 cases using the delta method took .61 seconds, while
computing the confidence intervals by bootstrap took 13 minutes 2 seconds for
1,000 replications.\footnote{Computations were made using Stata/SE 8.2 (born
10 Jan 2005) on a Dell XPS with an Intel$^{{\tiny \textregistered}}$
Pentium$^{{\tiny \textregistered}}$ 4 running at 3.4GHz. Estimates of times
for the delta method were based on the average of 1,000 computations.} Roughly
speaking, each replication in a bootstrap takes as long as the entire
computation for the delta method. The \texttt{zip} and \texttt{zinb} models
are too complex for computing the derivatives necessary for the delta method,
so only the bootstrap method is available.
\end{enumerate}

\noindent A summary of which methods work for which models is shown
below:\medskip\medskip%

\begin{tabular}
[c]{lccccc}\hline\hline
{\LARGE \ \ \ } &  & Endpoint &  &  & \\
& Maximum & Transfor- & Delta & Bootstrap & Default\\
\medskip Models & Likelihood (ML) & mations & Method & Method & Method\\\hline
\texttt{cloglog}, \texttt{logit}, \texttt{probit} & yes$^{1}$ & yes$^{2}$ &
yes & yes & delta\\
\texttt{ologit}, \texttt{oprobit} & yes$^{1}$ & no & yes & yes & delta\\
\texttt{mlogit}, \texttt{nbreg}, \texttt{poisson} & no & no & yes & yes &
delta\\
\texttt{zinb}, \texttt{zip} & no & no & no & yes & no ci\\
\texttt{regress}, \texttt{cnreg}, & ystar & no & no & no & ml\\
\texttt{\ \ \ \ \ intreg}, \texttt{tobit} & \multicolumn{1}{l}{} &
\multicolumn{1}{l}{} & \multicolumn{1}{l}{} & \multicolumn{1}{l}{} &
\multicolumn{1}{l}{}\\\hline\hline
\multicolumn{6}{l}{{\small 1: when the \texttt{ystar} option is specified; 2:
this method does not work for changes in predictions.}}%
\end{tabular}
\bigskip

\noindent We now discuss how to install \texttt{prvalue} and \texttt{prgen},
followed by a discussion of the syntax and options for each command.

\section{Installation}

\texttt{prvalue} and \texttt{prgen} are part of the \textsf{SPost} package of
programs for postestimation analysis (see Long and Freese forthcoming and
www.indiana.edu/\symbol{126}jslsoc/spost.htm for details). Installation should
be simple if you are connected to the Internet. If you have installed a prior
version of \textsf{SPost}, we suggest that you begin by uninstalling it. To do
this, enter the command \texttt{ado}. This will list all packages that have
been installed, with each package marked with a number in [ ]'s. Scan the list
and record the number of the package containing the \textsf{SPost} ado files
and any other packages related to \textsf{SPost}. While the ado files for
Stata 9 are contained in the package \texttt{spost\_ado9}, other names were
used in the past. Uninstall these packages with the command \texttt{ado
uninstall [}\#\texttt{]}, where \# is the package number (note that you must
include the brackets, for example, \texttt{ado uninstall [3]}). You must run
\texttt{ado uninstall} once for each package to be uninstalled.

Next, type \texttt{search} \texttt{spost, net} to get a list of packages
related to \textsf{SPost}. One of these packages should be labeled
\texttt{spost\_ado9 from http://www.indiana.edu/\symbol{126}jslsoc/stata}.
This contains the commands for Stata 9 and later. The package for Stata 8 is
labeled \texttt{spostado from http://www.indiana.edu/\symbol{126}%
jslsoc/stata}. Click on the blue label for your version of Stata and you will
be given information on how to install the programs. To install examples on
using \texttt{prvalue} and \texttt{prgen}, click on the label

\noindent\texttt{spostci from http://www.indiana.edu/\symbol{126}%
jslsoc/stata}. These do and dta files will reproduce the examples in this
article and will include other examples of using these commands.

If you have do files that used commands from prior versions of \textsf{SPost},
they should continue to work as before. The only exception is that now
\texttt{prvalue, ystar} does not print predicted probabilities. To compute the
predicted probabilities, simply run the command without the \texttt{ystar}
option. The formatting of output has also changed to incorporate the
confidence intervals.

\section{Syntax for prvalue}%

%TCIMACRO{\TeXButton{Ragged right start->}{\begin{flushleft}}}%
%BeginExpansion
\begin{flushleft}%
%EndExpansion


\begin{description}
\item \texttt{prvalue }[\texttt{if}\textit{\ exp}] [\texttt{in}%
\textit{\ range}] [\texttt{,} \texttt{x(}\textit{variable1}\texttt{\textit{=}%
}\textit{value1}[...]\texttt{) \underline{\texttt{r}}est(}\textit{stat}%
\texttt{) all \underline{\texttt{s}}ave \underline{\texttt{d}}iff
\underline{\texttt{max}}cnt(}\#\texttt{) \underline{\texttt{b}}rief
\underline{\texttt{noba}}se \underline{\texttt{nola}}bel \underline
{\texttt{ys}}tar \underline{\texttt{le}}vel(}\#\texttt{) }[ \texttt{\underline
{\texttt{del}}ta}
%TCIMACRO{\TEXTsymbol{\vert} }%
%BeginExpansion
$\vert$
%EndExpansion
\texttt{ept}
%TCIMACRO{\TEXTsymbol{\vert} }%
%BeginExpansion
$\vert$
%EndExpansion
\texttt{\underline{\texttt{boot}}strap} ]\texttt{\ reps(}\#\texttt{) dots
match size(}\#\texttt{) \underline{\texttt{sa}}ving(}...\texttt{)}%
][\ \texttt{\underline{\texttt{bias}}corrected}
%TCIMACRO{\TEXTsymbol{\vert} }%
%BeginExpansion
$\vert$
%EndExpansion
\texttt{\underline{\texttt{percent}}ile}
%TCIMACRO{\TEXTsymbol{\vert} }%
%BeginExpansion
$\vert$
%EndExpansion
\texttt{\underline{\texttt{norm}}al}]
\end{description}

%

%TCIMACRO{\TeXButton{<-ragged right end.}{\end{flushleft}}}%
%BeginExpansion
\end{flushleft}%
%EndExpansion


\paragraph{Options}

\begin{description}
\item \texttt{x(}\textit{variable1}\texttt{\textit{=}}\textit{value1}%
[...]\texttt{)} sets the values of independent variables for calculating
predicted outcomes. The list must alternate variable names and either numeric
values or terms describing the value to use. \texttt{mean}, \texttt{median},
\texttt{min}, and \texttt{max} are used to specify the mean, median, minimum
value or maximum value for a variable. \texttt{upper} sets a variables to its
minimum or maximum depending on which value yields the larger predicted value;
\texttt{lower} sets a variables to its minimum or maximum depending on which
value yields the smaller predicted value. If \texttt{prvalue} has already been
run, \texttt{previous} will set variables to their prior values. For example,
\texttt{x(wc=1 age=median)} or \texttt{x(wc=1 hc=previous age=mean age=40)}.

\item \texttt{rest(}\textit{stat}\texttt{)} sets the independent variables not
specified by \texttt{x()} to their \texttt{mean}, \texttt{min}, \texttt{max},
or \texttt{median} when calculating predicted values. \texttt{grmean} sets
these independent variables to the mean conditional on the variables and
values specified in \texttt{x()}; \texttt{grmedian}, \texttt{grmax}, and
\texttt{grmin} work in the same way using values for the median, maximum or
minimum. If \texttt{rest()} is not specified, \texttt{rest(mean)} is assumed.

\item \texttt{if}\textit{\ exp} and \texttt{in}\textit{\ range} specify the
sample used to compute the statistics (e.g., \texttt{mean}) used to set the
values with \texttt{x()} and \texttt{rest()}.

\item \texttt{all }specifies that calculation of means, medians, and other
statistics should use the entire sample instead of the possibly smaller sample
used to estimate the model.

\item \texttt{\texttt{s}ave }saves information from the current
\texttt{prvalue} for use in computing changes in predictions using the
\texttt{diff} option.

\item \texttt{\texttt{d}iff }computes difference between current predictions
and those that were saved using \texttt{save}. You must use the same method
for computing confidence intervals with \texttt{diff} as was used for the
\texttt{save} results.

\item \texttt{ystar }requests that the predicted value of $y^{\ast}$ rather
than predicted probabilities should be computed for binary and ordinal models.

\item \texttt{\texttt{max}cnt(}\#\texttt{) }is the maximum count value for
which the probability is computed in count models. The default of 9 will list
probabilities for values from 0 to 9.

\item \texttt{\texttt{b}rief }prints only limited output showing predictions
and confidence intervals, without listing values of the independent variables.

\item \texttt{\texttt{noba}se }suppresses the listing of the base values that
were specified with \texttt{x()} or \texttt{rest()}.

\item \texttt{\texttt{nola}bel }uses the numeric values of the outcome rather
than value labels in the output.
\end{description}

\paragraph{Options for computing confidence intervals}

\begin{description}
\item \texttt{level(}\#\texttt{) }specifies the confidence level, in percent,
for confidence intervals. For example, \texttt{level(95)} specifies that you
want a 95\% confidence interval.
\end{description}

\noindent One of the following options can be chosen to specify the method
used to compute confidence intervals:

\begin{description}
\item \texttt{ept }computes confidence intervals for predicted probabilities
for \texttt{cloglog}, \texttt{logit} and \texttt{probit} by endpoint
transformation. This method cannot be used for changes in predictions.

\item \texttt{\texttt{del}ta }calculates confidence intervals by the delta
method using analytic derivatives.

\item \texttt{\texttt{boot}strap }computes confidence intervals using the
bootstrap method. This method takes roughly 1,000 times longer to compute than
other methods.
\end{description}

\paragraph{Options used for bootstrapped confidence intervals}

\begin{description}
\item \texttt{reps(}\#\texttt{) }specifies the number of bootstrap
replications to be performed. The default is 1,000. The accuracy of a
bootstrap estimate depends critically on the number of replications. While
sources differ on the recommended number of replications, Efron and Tibshirani
(1993:188) suggest 1,000 replications for confidence intervals. You can use
\texttt{bssize} (Poi 2004) to calculate the number of bootstrap replications
to be used. In our experience, this method suggests over 1,000 replications.
See the documentation for \texttt{saving} below.

\item \texttt{dots }is used with \texttt{bootstrap} to write a . at the
beginning of each replication and periodically prints the percent of total
replications that has been completed. If computations appears to be stuck
(i.e., new dots do not appear), it is likely that the estimation is not
converging for the current bootstrap sample. We have found this to be most
common with \texttt{zip} and \texttt{zinb}. When this happens, you can click
on the break symbol to stop computations for the current sample or wait until
the maximum number of iterations have been computed (by default, the maximum
number of iterations is 16,000). When a model does not converge for a given
bootstrap sample, that sample is dropped.

\item \texttt{match }specifies that the bootstrap will resample within each
category of the dependent variable in proportion to the distribution of the
outcome categories in the estimation sample. If \texttt{match} is not
specified, the proportions in each category of the bootstrap sample are
determined entirely by the random draw and it is possible to have samples with
no cases in some of the categories. This option does not apply to
\texttt{cnreg}, \texttt{intreg}, \texttt{nbreg}, \texttt{poisson},
\texttt{regress}, \texttt{tobit}, \texttt{zinb}, and \texttt{zip}.

\item \texttt{size(}\#\texttt{) }specifies the number of cases to be sampled
when bootstrapping. The default is the size of the estimation sample. If
\texttt{size(}\#\texttt{)} is specified, \# must be less than or equal to the
size of the estimation sample. In general, it is best to not specify
\texttt{size} (see www.stata.com/support/faqs/stat/reps.html for further information).

\item \texttt{saving(}\textit{filename, save\_options}\texttt{)}%
\textbf{\ }creates a data file with the estimates from each of the
bootstrapped samples with one case for each replication. This option is useful
when you need to examine the distribution of bootstrapped estimates. For
example, this option is required if you plan to use \texttt{bssize} to
calculate the number of replications to be used (Poi 2004).
\end{description}

\noindent One of the following can be chosen if you are computing bootstrapped
confidence intervals:

\begin{description}
\item \texttt{\texttt{percent}ile} computes the bootstrapped confidence
interval using the percentile method. This is the default method.

\item \texttt{\texttt{bias}corrected} computes the bootstrapped confidence
interval using the bias-corrected method.

\item \texttt{\texttt{norm}al} computes the bootstrapped confidence interval
using the normal approximation method.
\end{description}

\section{Syntax for prgen}%

%TCIMACRO{\TeXButton{Ragged right start->}{\begin{flushleft}}}%
%BeginExpansion
\begin{flushleft}%
%EndExpansion


\begin{description}
\item \texttt{prgen }\textit{varname}\ [\texttt{if }\textit{exp}] [\texttt{in
}\textit{range}]\texttt{, \underline{\texttt{g}}enerate(}\textit{prefix}%
\texttt{) }[\texttt{\underline{\texttt{f}}rom(}\#\texttt{) \underline
{\texttt{t}}o(}\#\texttt{) \underline{\texttt{n}}cases(}\#\texttt{)
gap(}\#\texttt{) \underline{\texttt{x}}(}\textit{variable1}\texttt{=}%
\textit{value1 }[...]\texttt{) \underline{\texttt{r}}est(}\textit{stat}%
\texttt{) \underline{\texttt{m}}axcnt(}\#\texttt{) \underline{\texttt{b}}rief
\underline{\texttt{a}}ll \underline{\texttt{noi}}sily \underline{\texttt{mar}%
}ginal ci}][ options from \texttt{prvalue}\textit{\ }]
\end{description}

\medskip\noindent All options from \texttt{prvalue}, except for \texttt{save}
and \texttt{diff}, can be used to specify how the upper and lower bounds are
created when the \texttt{ci} option is used. The options \texttt{if},
\texttt{in}, \texttt{x()}, \texttt{rest()}, \texttt{maxcnt}, \texttt{brief}
and \texttt{all} work in the same way as for \texttt{prvalue}.%

%TCIMACRO{\TeXButton{<-ragged right end.}{\end{flushleft}}}%
%BeginExpansion
\end{flushleft}%
%EndExpansion


\paragraph{Options}

\begin{description}
\item \textit{varname} is the name of the variable that changes while all
other variables are held at specified values.

\item \texttt{\texttt{g}enerate(}\textit{prefix}\texttt{)} sets the prefix for
the new variables created by \texttt{prgen}. Choosing a prefix that is
different than the beginning letters of any of the variables in your data set
makes it easier to examine the results. For example, if you choose the prefix
\texttt{abcd} then you can use the command \texttt{sum abcd*} to examine all
newly created variables.

\item \texttt{\texttt{f}rom(}\#\texttt{)} and \texttt{\texttt{to}%
(}\#\texttt{)} are the start and end values for \textit{varname}. The default
is for \textit{varname }to range from the observed minimum to the observed
maximum of \textit{varname}.

\item \texttt{\texttt{n}cases(}\#\texttt{)} specifies the number of predicted
values \texttt{prgen} computes as \textit{varname} varies from the start value
to the end value. The default is 11.

\item \texttt{gap(}\#\texttt{)} is an alternative to \texttt{ncases}. You
specify the gap or size between tic marks and \texttt{prgen} determines if the
specified value divides evenly into the range specified with \texttt{from} and
\texttt{to}. If it does, \texttt{prgen} determines the appropriate value for
\texttt{ncases}.

\item \texttt{ci }indicates that you want to generate confidence intervals
corresponding to the predictions that are created.

\item \texttt{marginal} requests that a variable or variables are created
containing the marginal change in the outcome relative to \textit{varname},
holding all other variables constant.

\item \texttt{noisily} indicates that you want to see the output from
\texttt{prvalue} that was used to generate the predicted values.
\end{description}

\paragraph{Variables generated}

The following table indicates the variables that are created by \texttt{prgen}%
. The observations contain predicted values or probabilities for a range of
values for the variable \textit{varname}, holding the other variables at the
specified values. $n$ observations are created, where $n$ is 11 by default or
can be specified by \texttt{ncases()}or \texttt{gap()}. The new variable names
start with the\textit{\ prefix }specified by \texttt{gen()}. The variables
created are:\bigskip%

%TCIMACRO{\TeXButton{footnotesize}{\footnotesize}}%
%BeginExpansion
\footnotesize
%EndExpansion


\noindent%
\begin{tabular}
[c]{lll}\hline\hline
For which models{\LARGE \ \ \ } & Name & Content\\\hline
All models{\LARGE \ \ \ } & \textit{prefix}\texttt{x} & Values of
\textit{varname}.\\
\texttt{logit}, \texttt{probit} & \textit{prefix}\texttt{p0} & Predicted
probability $\Pr(y=0).$\\
& \textit{prefix}\texttt{p1} & Predicted probability $\Pr(y=1).$\\
\texttt{ologit}, \texttt{oprobit} & \textit{prefix}\texttt{p}\emph{k} &
Predicted probability $\Pr(y=k)$ for all outcomes.\\
& \textit{prefix}\texttt{s}\emph{k} & Cumulative probability $\Pr(y\leq k)$
for all outcomes.\\
\texttt{mlogit} & \textit{prefix}\texttt{p}\emph{k} & Predicted probability
$\Pr(y=k)$ for all outcomes.\\
\texttt{poisson}, \texttt{nbreg}, \texttt{zip}, \texttt{zinb} &
\textit{prefix}\texttt{mu} & Predicted rate $\mu.$\\
& \textit{prefix}\texttt{p}\emph{k} & Predicted probability $\Pr(y=k)$, for
$0\leq k\leq$ \texttt{maxcnt()}.\\
& \textit{prefix}\texttt{s}\emph{k} & Cumulative probability $\Pr(y\leq k)$,
for $0\leq k\leq$ \texttt{maxcnt()}.\\
\texttt{zip}, \texttt{zinb} & \textit{prefix}\texttt{inf} & Predicted
probability $\Pr($Always 0$=1)=\Pr\left(  \text{\textit{inflate}}\right)  .$\\
\texttt{regress}, \texttt{tobit}, \texttt{cnreg}, \texttt{intreg} &
\textit{prefix}\texttt{xb} & Predicted value $\mathbf{x}\widehat
{\boldsymbol{\beta}}\mathbf{.}$\\\hline\hline
\end{tabular}
%

%TCIMACRO{\TeXButton{normalsize}{\normalsize}}%
%BeginExpansion
\normalsize
%EndExpansion


\medskip\noindent If \texttt{ci} is specified as an option for \texttt{prgen},
variables are created containing the upper and lower bounds for the confidence
interval for the outcome. These variables have the same names as those in the
table above, except for adding \texttt{ub} at the end for the variable with
the upper bound and \texttt{lb} for the lower bound. If \texttt{marginal} is
specified, variables are created that contain the marginal change in the
outcome with respect to \textit{varname}, holding all other variables
constant. The variables containing marginals have the same names as those in
the table above, except for adding a \texttt{D} prior to the outcome
abbreviation and \texttt{D}\textit{varname} after. For example, the marginal
for \textit{prefix}\texttt{p0} is named \textit{prefix}\texttt{Dp0D}%
\textit{varname.} Marginals are only computed for those models for which the
\texttt{prchange} command can compute the marginal change (see Long and Freese forthcoming).

\section{Examples}

The examples that follow illustrate increasingly complex ways in which
\texttt{prvalue} can be used. In addition to these examples, other examples
are downloaded when you install the \texttt{spostci} package. For further
details on the data and models, see either Long (1997) or Long and Freese (forthcoming).

\subsection{Simple predictions}

The commands generating the output in this section are in
\texttt{prvalue\_predict.do} in the \texttt{spostci} package.

\paragraph{Binary logit}

In this example, we use a binary logit model to predict labor force
participation for a sample of women. The independent variables are the number
of children less than six, the number of children from six to 18, the wife's
age, whether the wife went to college, whether the husband went to college,
the log of the wife's estimated wages, and the family income excluding wife's income.
\begin{verbatim}
. use binlfp2, clear
. logit lfp k5 k618 age wc hc lwg inc, nolog
\end{verbatim}

(\textit{output omitted})\medskip

\noindent After the model is estimated, we compute the predicted probability
of labor force participation for a woman who is 35 years old (\texttt{age=35}%
), has two young children (\texttt{k5=2}), did not attend college
(\texttt{wc=0}), and is average on all other characteristics
(\texttt{rest(mean)}):
\begin{verbatim}
. prvalue, x(age=35 k5=2 wc=0) rest(mean)
 
logit: Predictions for lfp
 
Confidence intervals by delta method
 
                                95% Conf. Interval
  Pr(y=inLF|x):       0.1174   [ 0.0495,    0.1852]
  Pr(y=NotInLF|x):    0.8826   [ 0.8148,    0.9505]
 
           k5       k618        age         wc         hc        lwg        inc
x=          2  1.3532537         35          0  .39176627  1.0971148  20.128965
\end{verbatim}

\noindent For someone with these characteristics (which are listed in the line
beginning \texttt{x=}), the predicted probability of being in the labor force
is .117, with a 95\% confidence interval from .050 to .185.

With binary and ordinal models, we can also predict the latent $y^{\ast}$ used
in the latent variable formulation of the model. To do this, we add the option
\texttt{ystar}:
\begin{verbatim}
. prvalue , x(age=35 k5=2 wc=0) rest(mean) ystar
 
logit: Predictions for lfp
 
                                95% Conf. Interval
  Predicted y*:      -2.0177   [-2.6723,   -1.3631]
 
           k5       k618        age         wc         hc        lwg        inc
x=          2  1.3532537         35          0  .39176627  1.0971148  20.128965 
\end{verbatim}

\paragraph{Negative binomial regression}

In this example, we examine a count model predicting the number of papers
published by biochemists.\ Independent variables are whether the scientist is
a woman, whether the scientist is married, the number of young children in the
scientist's family, the prestige of scientist's Ph.D. department and the
number of articles published by the scientist's mentor.
\begin{verbatim}
. use couart2, clear
. nbreg art fem mar kid5 phd ment, nolog
\end{verbatim}

(\textit{output omitted})\medskip

\noindent Since very few scientists publish more than six papers, we use the
option \texttt{maxcnt(6)} so that \texttt{prvalue} will only compute predicted
probabilities for publications from 0 to 6. For a single (\texttt{mar=0})
woman (\texttt{fem=0}) without young children (\texttt{kid5=0}) who is average
on all other characteristics, we find:
\begin{verbatim}
. prvalue , x(mar=0 fem=1 kid5=0) rest(mean) maxcnt(6)
 
nbreg: Predictions for art
 
Confidence intervals by delta method
 
                                95% Conf. Interval
  Rate:               1.4079   [ 1.2237,    1.5921]
  Pr(y=0|x):          0.3346   [ 0.2966,    0.3726]
  Pr(y=1|x):          0.2905   [ 0.2809,    0.3000]
  Pr(y=2|x):          0.1818   [ 0.1731,    0.1904]
  Pr(y=3|x):          0.0991   [ 0.0863,    0.1118]
  Pr(y=4|x):          0.0500   [ 0.0395,    0.0604]
  Pr(y=5|x):          0.0240   [ 0.0170,    0.0310]
  Pr(y=6|x):          0.0111   [ 0.0070,    0.0153]
 
          fem        mar       kid5        phd       ment
x=          1          0          0  3.1031093  8.7672131
\end{verbatim}

\noindent We predict 1.41 publications, with a confidence interval from 1.22
to 1.59. Below the rate, the probabilities that a person with these
characteristics will publish a given number of papers are listed along with
confidence intervals computed with the delta method.

\paragraph{Predictions at observed values}

\texttt{prvalue} is designed to compute predictions at values specified by the
user (e.g., \texttt{x(k5=0)}) or at values corresponding to summary statistics
in the sample (e.g., \texttt{rest(mean)}). If you want to compute predictions
along with confidence intervals at observed values for a given observation,
you need to explicitly indicate those values. For example, the first
observation in the data used with \texttt{nbreg} is:
\begin{verbatim}
. list in 1
 
     +------------------------------------------+
     | art   fem       mar   kid5    phd   ment |
     |------------------------------------------|
  1. |   0   Men   Married      0   2.52      7 |
     +------------------------------------------+
\end{verbatim}

\noindent To compute the predicted outcomes at these values, we can use the command:
\begin{verbatim}
. prvalue, x(fem=0 mar=1 kid5=0 phd=2.52 ment=7)
 
nbreg: Predictions for art
 
Confidence intervals by delta method
 
                                95% Conf. Interval
  Rate:                1.913   [ 1.6587,    2.1673]
  Pr(y=0|x):          0.2499   [ 0.2155,    0.2843]
  Pr(y=1|x):          0.2591   [ 0.2421,    0.2762]
  Pr(y=2|x):          0.1937   [ 0.1925,    0.1949]
  Pr(y=3|x):          0.1261   [ 0.1162,    0.1360]
  Pr(y=4|x):          0.0760   [ 0.0646,    0.0874]
  Pr(y=5|x):          0.0436   [ 0.0339,    0.0533]
  Pr(y=6|x):          0.0242   [ 0.0171,    0.0313]
  Pr(y=7|x):          0.0131   [ 0.0083,    0.0179]
  Pr(y=8|x):          0.0069   [ 0.0039,    0.0100]
  Pr(y=9|x):          0.0036   [ 0.0018,    0.0055]
 
     fem   mar  kid5   phd  ment
x=     0     1     0  2.52     7
\end{verbatim}

\noindent Using a bit of simple Stata programming, it is possible to automate
this process to compute predictions for all observations. This is shown in the
sample do file \texttt{pravlue\_observed.do} that is installed as part of the
\texttt{spostci} package.

\subsection{Discrete change}

The commands generating the output in this section are in
\texttt{prvalue\_change.do} in the \texttt{spostci} package.

\paragraph{Binary probit}

One way to interpret the results of regression type models is to see how the
predictions change when levels of the independent variables change. To
illustrate this, we examine how the probability of being in the labor force is
expected to increase when a woman has gone to college. We begin by estimating
a probit model:
\begin{verbatim}
. use binlfp2, clear
. probit lfp k5 k618 age wc hc lwg inc, nolog
\end{verbatim}

(\textit{output omitted})\medskip

\noindent The next command predicts the probability for someone who is average
on all characteristics and who has not gone to college (\texttt{wc=0}). Notice
that this command is prefixed by \texttt{quietly} since we do not want to see
the results yet. But, we save the results with the \texttt{save} option.
\begin{verbatim}
. quietly prvalue, x(wc=0) rest(mean) save 
\end{verbatim}

\noindent We run \texttt{prvalue} again, this time for someone who attended
college (\texttt{wc=1}). We use the \texttt{diff} option to compute changes in
predictions from the saved results:
\begin{verbatim}
. prvalue, x(wc=1) rest(mean) diff
 
probit: Change in Predictions for lfp
 
Confidence intervals by delta method
 
                     Current     Saved    Change   95% CI for Change
  Pr(y=inLF|x):       0.7082    0.5238    0.1844  [ 0.0892,   0.2795]
  Pr(y=NotInLF|x):    0.2918    0.4762   -0.1844  [-0.2795,  -0.0892]
 
                 k5       k618        age         wc         hc        lwg
Current=   .2377158  1.3532537  42.537849          1  .39176627  1.0971148
  Saved=   .2377158  1.3532537  42.537849          0  .39176627  1.0971148
   Diff=          0          0          0          1          0          0
 
                inc
Current=  20.128965
  Saved=  20.128965
   Diff=          0
\end{verbatim}

\noindent For someone who is average on all other variables, attending college
increases the probability of being in the labor force by .18 with a 95\%
confidence interval from .09 to .28.

\paragraph{Poisson regression}

Our next example computes the change in the rate of publication when two
variables change at the same time. We also illustrate the computation of
bootstrapped confidence intervals.
\begin{verbatim}
. use couart2, clear
. poisson art fem mar kid5 phd ment, nolog
\end{verbatim}

(\textit{output omitted})\medskip

\noindent We want to compare the predicted productivity for an unmarried woman
without children to the productivity for a married, female scientist with two
young children:
\begin{verbatim}
. quietly prvalue , x(mar=0 fem=1 kid5=0) maxcnt(5) boot save
 
. prvalue , x(mar=1 fem=1 kid5=2) maxcnt(5) boot diff
 
poisson: Change in Predictions for art
 
Bootstrapped confidence intervals using percentile method
(1000 of 1000 replications completed)
 
                     Current     Saved    Change   95% CI for Change
  Rate:                1.138    1.4102   -.27228  [-0.5110,  -0.0300]
  Pr(y=0|x):          0.3205    0.2441    0.0764  [ 0.0080,   0.1535]
  Pr(y=1|x):          0.3647    0.3442    0.0205  [ 0.0022,   0.0384]
  Pr(y=2|x):          0.2075    0.2427   -0.0352  [-0.0728,  -0.0034]
  Pr(y=3|x):          0.0787    0.1141   -0.0354  [-0.0652,  -0.0039]
  Pr(y=4|x):          0.0224    0.0402   -0.0178  [-0.0327,  -0.0020]
  Pr(y=5|x):          0.0051    0.0113   -0.0062  [-0.0121,  -0.0007]
 
                fem        mar       kid5        phd       ment
Current=          1          1          2  3.1031093  8.7672131
  Saved=          1          0          0  3.1031093  8.7672131
   Diff=          0          1          2          0          0
\end{verbatim}

\noindent By default, 1,000 replications are computed, all of which completed
successfully as indicated by the count of the number of replications completed.

\subsection{Plotting confidence intervals}

The commands generating the output in this section are in
\texttt{prgen\_plotpred.do} in the \texttt{spostci} package.

\paragraph{Predicted probabilities with logit}

\texttt{prgen} makes it simple to plot predictions and confidence bands for
those predictions as one variable changes, holding all other variables
constant. First, we estimate a binary logit model:
\begin{verbatim}
. use binlfp2, clear
. logit lfp k5 k618 age wc hc lwg inc, nolog
\end{verbatim}

(\textit{output omitted})\medskip

\noindent We want to plot the predicted probability of being in the labor
force for average women at various ages. The resulting plot looks like this:%

%TCIMACRO{\FRAME{dtbpF}{6.0502in}{4.0421in}{0pt}{}{}{prgen_plotpred_logit.wmf}%
%{\special{ language "Scientific Word";  type "GRAPHIC";
%maintain-aspect-ratio TRUE;  display "USEDEF";  valid_file "F";
%width 6.0502in;  height 4.0421in;  depth 0pt;  original-width 15.2397in;
%original-height 10.1598in;  cropleft "0";  croptop "1";  cropright "1";
%cropbottom "0";  filename 'prgen_plotpred_logit.wmf';file-properties "XNPEU";}%
%}}%
%BeginExpansion
\begin{center}
\includegraphics[
natheight=10.159800in,
natwidth=15.239700in,
height=4.0421in,
width=6.0502in
]%
{prgen_plotpred_logit.wmf}%
\end{center}
%EndExpansion
As would be expected, the probability of being in the labor force decreases
with age and the confidence interval narrows at the center of our data. To
create this graph, we use \texttt{prgen} to compute predictions at many
different values of \texttt{age} and to store the predictions. The command
that follows includes several options governing the confidence interval.
First, \texttt{ci} tells \texttt{prgen} that you want to save values for
plotting the confidence interval. The \texttt{ept} option indicates that the
confidence interval should be computed using endpoint transformations, rather
than the default which is the delta method. Finally, \texttt{noisily}
indicates that you want to see the results of \texttt{prvalue} each time it is
used to compute a prediction and its confidence interval; in practice, you
would only use \texttt{noisily} if you were having problems getting the
results to converge. Using \texttt{prgen}, we now compute predictions as
\texttt{age} ranges from 20 to 70 in steps of five years:
\begin{verbatim}
. prgen age, from(20) to(70) gap(5) gen(prlfp) ci ept noisily
 
Results from prvalue called by prgen
 
logit: Predictions for lfp
 
                                95% Conf. Interval
  Pr(y=inLF|x):       0.8495   [ 0.7565,    0.9111]
  Pr(y=NotInLF|x):    0.1505   [ 0.0889,    0.2435]
 
           k5       k618        age         wc         hc        lwg        inc
x=   .2377158  1.3532537         20   .2815405  .39176627  1.0971148  20.128965
\end{verbatim}

(\textit{output omitted})\medskip\ 
\begin{verbatim}
logit: Predicted values as age varies from 20 to 70.
 
           k5       k618        age         wc         hc        lwg        inc
x=   .2377158  1.3532537  42.537849   .2815405  .39176627  1.0971148  20.128965
\end{verbatim}

\noindent The variables that are created all begin with the prefix
\texttt{prlfp}:
\begin{verbatim}
. desc prlfp*
 
              storage  display     value
variable name   type   format      label      variable label
-------------------------------------------------------------------------------
prlfpx          float  %9.0g                  Changing value of age
prlfpp0         float  %9.0g                  pr(NotInLF)=Pr(0)
prlfpp1         float  %9.0g                  pr(inLF)=Pr(1)
prlfpp0ub       float  %9.0g                  UB pr(NotInLF)=Pr(0)
prlfpp1ub       float  %9.0g                  UB pr(inLF)=Pr(1)
prlfpp0lb       float  %9.0g                  LB pr(NotInLF)=Pr(0)
prlfpp1lb       float  %9.0g                  LB pr(inLF)=Pr(1)
\end{verbatim}

\noindent\texttt{prlfpx} contains values of \texttt{age} between 20 and 70.
\texttt{prlfpp1} is the probability of being in the labor force, while
\texttt{prlfpp1ub}\ and \texttt{prlfpp1lb}\ hold the upper and lower bounds.
We can plot these with the following commands, which generated the graph
above. First, we create variable labels that are used to label the plot:
\begin{verbatim}
. label var prlfpp1 "Predicted probability"
. label var prlfpp1ub "95% upper limit"
. label var prlfpp1lb "95% lower limit"
. label var prlfpx "Age"
\end{verbatim}

\noindent Next, we use \texttt{twoway} to plot the results:
\begin{verbatim}
. twoway ///
>     (connected prlfpp1 prlfpx, clcolor(black) clpat(solid) ///
>         clwidth(medthick) msymbol(i) mcolor(none))         ///
>     (connected prlfpp1ub prlfpx, msymbol(i) mcolor(none)   ///
>         clcolor(black) clpat(dash) clwidth(thin))          ///
>     (connected prlfpp1lb prlfpx, msymbol(i) mcolor(none)   ///
>         clcolor(black) clpat(dash) clwidth(thin)),         ///
>     ytitle("Probability of Being in Labor Force")          ///
>     yscale(range(0 .35)) ylabel(, grid glwidth(medium)     ///
>     glpattern(dot)) xscale(range(20 70)) xlabel(20(10)70)
\end{verbatim}

\subsection{Effects of the number of replications}

The number of replications is important for the quality of the bootstrapped
confidence interval. To illustrate this, we consider our earlier example of
labor force participation:
\begin{verbatim}
use binlfp2, clear
logit lfp k5 k618 age wc hc lwg inc, nolog
\end{verbatim}

\noindent For this model, we compute the 95\% confidence interval using the
bootstrap method with the number of replications varying from 50 to 3,000.
This leads to the following graph, which shows that the upper and lower bounds
from the bootstrap vary substantially when the number of replications is less
than 1,000 (especially for the upper bounds), but stabilize after 1,000.
%TCIMACRO{\FRAME{dtbpF}{6.0502in}{3.8908in}{0pt}{}{}{prvalue_boot_reps.wmf}%
%{\special{ language "Scientific Word";  type "GRAPHIC";
%maintain-aspect-ratio TRUE;  display "USEDEF";  valid_file "F";
%width 6.0502in;  height 3.8908in;  depth 0pt;  original-width 15.2397in;
%original-height 10.1598in;  cropleft "0";  croptop "1";  cropright "1";
%cropbottom "0.0375";
%filename 'prvalue_boot_reps.wmf';file-properties "XNPEU";}}}%
%BeginExpansion
\begin{center}
\includegraphics[
trim=0.000000in 0.380993in 0.000000in 0.000000in,
natheight=10.159800in,
natwidth=15.239700in,
height=3.8908in,
width=6.0502in
]%
{prvalue_boot_reps.wmf}%
\end{center}
%EndExpansion


Here is how the graph was generated. First, we compute the predicted
probability of being in the labor force with \texttt{inc} set to 100 and other
variables held at the mean, with the 95\% confidence interval computed by the
delta method:
\begin{verbatim}
prvalue, x(inc=100) rest(mean) delta
local PrLFP = pepred[2,2]
local UpperDelta = peupper[2,2]
local LowerDelta = pelower[2,2]
\end{verbatim}

\noindent The values of predicted probability along with the upper and lower
bounds are saved to the local variables \texttt{PrLFP}, \texttt{UpperDelta}%
\ and \texttt{LowerDelta}\ that are used to plot the sold line for the
predicted probability at 0.080 and the dashed lines for the lower and upper
bounds at -0.015 and 0.176 in the graph above). Next, we we create variables
that will hold the values for the number of bootstrap replications used and
the bounds computed using a given number of replications. These variables are
plotted to create our graph:
\begin{verbatim}
gen reps = .
    label var reps "# of replications"
gen ubboot = .
    label var ubboot "Bootstrap: Upper bound"
gen lbboot = .
    label var lbboot "Bootstrap: Lower bound"
\end{verbatim}

\noindent Before computing the first bootstrapped confidence interval, we set
the random number seed so that we can reproduce our results later (without
this, a new sequence of random numbers are generated each time we run the
program). We then loop through values for the number of replications ranging
from 50 to 3000. Within the loop, we compute bootstrapped confidence intervals
and save the values into the variables we created above:
\begin{verbatim}
set seed 2399194
local j = 0
foreach reps in 50 100 200 300 400 500 600 700 800 900 1000 ///
        1200 1400 1600 1800 2000 2200 2400 2600 3000 {
    local ++j
    qui replace reps = `reps' if _n==`j'
    di "= Start for `reps' replications: " c(current_time)
    di "= Seed: " c(seed)
    prvalue , x(inc=100) rest(mean) boot rep(`reps')
    scalar UpperBoot = peupper[2,2]
    qui replace ubboot = UpperBoot if _n==`j'
    scalar LowerBoot = pelower[2,2]
    qui replace lbboot = LowerBoot if _n==`j'
    di
    di "= End for `reps' replications: " c(current_time)
    di
}
\end{verbatim}

\noindent The results were then plotted. Consistent with the recommendations
given above, we see that the bounds begin to stabilize at around 1,000
replications, which is our default. The commands generating the output in this
section are in \texttt{prvalue\_boot\_reps.do}.

\section{\label{CISection}Methods for Computing Confidence Intervals}

In this section we provide a more technical discussion of the ideas behind the
methods used by \texttt{prvalue} to compute confidence intervals. Let $\delta$
be some parameter estimated by your model (e.g., $\Pr\left(  y=1\right)  $ in
the logit model). We are interested in computing the lower and upper bounds
such that $\Pr\left(  \text{LB}\leq\delta\leq\text{UB}\right)  =\alpha\ .$This
can be interpreted as saying that the population parameter $\delta$ is
contained within the interval with confidence $1-\alpha$. In the simplest
case, the confidence interval can be computed directly using results from
maximum likelihood theory. Under the usual conditions for ML, $\widehat
{\boldsymbol{\beta}}\overset{a}{\sim}N\left(  \boldsymbol{\beta}%
,\text{\textit{Var}}\left(  \widehat{\boldsymbol{\beta}}\right)  \right)  $.
Then, $\mathbf{x}\widehat{\boldsymbol{\beta}}\overset{a}{\sim}N\left(
\mathbf{x}^{\prime}\boldsymbol{\beta},\mathbf{x}^{\prime}\text{\textit{Var}%
}\left(  \widehat{\boldsymbol{\beta}}\right)  \mathbf{x}\right)  $, which can
be used to compute confidence intervals for linear combinations $\mathbf{x}%
\widehat{\boldsymbol{\beta}}$, such as $\widehat{y}$ in linear regression. For
example, to compute the $100\left(  1-\alpha\right)  $ confidence interval,
define $z$ as the $\left(  1-\frac{\alpha}{2}\right)  $ percentile from a
standard normal distribution (i.e., the probability of being greater than $z$
is $\alpha/2$ and the probability of being less than $-z$ is $\alpha/2$).
Since $\mathbf{x}\widehat{\boldsymbol{\beta}}$ is asymptotically normal with
variance $\mathbf{x}^{\prime}$\textit{Var}$\left(  \widehat{\boldsymbol{\beta
}}\right)  \mathbf{x}$, the $100\left(  1-\alpha\right)  $ confidence interval
for $\mathbf{x}^{\prime}\boldsymbol{\beta}$ is%
\begin{equation}
\left(  \mathbf{x}\widehat{\boldsymbol{\beta}}-z\sqrt{\mathbf{x}^{\prime
}\text{\textit{Var}}\left(  \widehat{\boldsymbol{\beta}}\right)  \mathbf{x}%
}\right)  \leq\mathbf{x}^{\prime}\boldsymbol{\beta}\leq\left(  \mathbf{x}%
\widehat{\boldsymbol{\beta}}+z\sqrt{\mathbf{x}^{\prime}\text{\textit{Var}%
}\left(  \widehat{\boldsymbol{\beta}}\right)  \mathbf{x}}\right)  \ .
\label{mlci1}%
\end{equation}
Since many of the quantities computed by \texttt{prvalue} are nonlinear
transformations of $\mathbf{x}^{\prime}\boldsymbol{\beta}$, other methods are required.

\subsection{Endpoint Transformations}

The method of endpoint transformation can compute confidence intervals for
monotonic functions of $\mathbf{x}^{\prime}\boldsymbol{\beta}$ (i.e., a
function that is always increasing or always decreasing), such as the
predicted probabilities in binary logit or probit. First, the confidence
interval for $\mathbf{x}\widehat{\boldsymbol{\beta}}$ is computed as the
symmetric interval $\left[  \text{LB}_{\mathbf{x}^{\prime}\boldsymbol{\beta}%
}\leq\mathbf{x}^{\prime}\boldsymbol{\beta}\leq\text{UB}_{\mathbf{x}^{\prime
}\boldsymbol{\beta}}\right]  $. Since $\Pr\left(  \mathbf{x}^{\prime
}\boldsymbol{\beta}\right)  =F\left(  \mathbf{x}^{\prime}\boldsymbol{\beta
}\right)  $ is a monotonic transformation of $\mathbf{x}^{\prime
}\boldsymbol{\beta}$, the confidence interval for $F\left(  \mathbf{x}%
^{\prime}\boldsymbol{\beta}\right)  $ is computed by transforming the bounds
using the same function $F$:%
\[
\left[  \Pr\left(  \mathbf{x}^{\prime}\boldsymbol{\beta}\right)  _{\text{LB}%
}\leq\Pr\left(  \mathbf{x}^{\prime}\boldsymbol{\beta}\right)  \leq\Pr\left(
\mathbf{x}^{\prime}\boldsymbol{\beta}\right)  _{\text{UB}}\right]  =\left[
F\left(  \left[  \mathbf{x}^{\prime}\boldsymbol{\beta}\right]  _{\text{LB}%
}\right)  \leq F\left(  \mathbf{x}^{\prime}\boldsymbol{\beta}\right)  \leq
F\left(  \left[  \mathbf{x}^{\prime}\boldsymbol{\beta}\right]  _{\text{UB}%
}\right)  \right]  \ .
\]
For example, consider the binary logit model. Using equation \ref{mlci1}, we
compute the $100\left(  1-\alpha\right)  $ confidence interval for
$\mathbf{x}\widehat{\boldsymbol{\beta}}$. To compute the predicted
probability, we take the logit transformation of $\mathbf{x}\widehat
{\boldsymbol{\beta}}$:%
\[
\Pr\left(  \mathbf{x}\widehat{\boldsymbol{\beta}}\right)  =\frac{\exp\left(
\mathbf{x}\widehat{\boldsymbol{\beta}}\right)  }{1+\exp\left(  \mathbf{x}%
\widehat{\boldsymbol{\beta}}\right)  }=\Lambda\left(  \mathbf{x}%
\widehat{\boldsymbol{\beta}}\right)  \ ,
\]
where $\Lambda\left(  .\right)  $ is the cdf for the logistic distribution. By
applying the logit transformation to the endpoints from equation \ref{mlci1},
we obtain the asymmetric confidence interval for $\Pr\left(  \mathbf{x}%
^{\prime}\boldsymbol{\beta}\right)  $:%
\[
\Lambda\left(  \mathbf{x}\widehat{\boldsymbol{\beta}}-z\sqrt{\mathbf{x}%
^{\prime}\mathit{Var}\left(  \widehat{\boldsymbol{\beta}}\right)  \mathbf{x}%
}\right)  \leq\Pr\left(  \mathbf{x}^{\prime}\boldsymbol{\beta}\right)
\leq\Lambda\left(  \mathbf{x}\widehat{\boldsymbol{\beta}}+z\sqrt
{\mathbf{x}^{\prime}\mathit{Var}\left(  \widehat{\boldsymbol{\beta}}\right)
\mathbf{x}}\right)  \ .
\]
While computationally simple, this method is limited to those few cases in
which the outcome of interest is a monotonic function of $\mathbf{x}^{\prime
}\boldsymbol{\beta}$. For further discussion, see Cox and Ma (1995) and Liao (2000).

\subsection{Delta Method}

The delta method is a more general method for computing confidence intervals.
This method takes a function that is too complex for analytically computing
the variance (e.g., \textit{Var}$\left(  \exp\left(  \mathbf{x}\widehat
{\boldsymbol{\beta}}\right)  \left[  1+\exp\left(  \mathbf{x}\widehat
{\boldsymbol{\beta}}\right)  \right]  ^{-1}\right)  $, creates a linear
approximation of the function, and then computes the variance of the simpler
linear function that is used for large sample inference. While we illustrate
this approach with a simple, one-parameter example, the approach generalizes
readily to the case with multiple parameters. Details on the equations used to
implement the delta method for the models in \texttt{prvalue} are available at
www.indiana.edu%
%TCIMACRO{\TEXTsymbol{\backslash}}%
%BeginExpansion
$\backslash$%
%EndExpansion
\symbol{126}jslsoc%
%TCIMACRO{\TEXTsymbol{\backslash}}%
%BeginExpansion
$\backslash$%
%EndExpansion
stata%
%TCIMACRO{\TEXTsymbol{\backslash}}%
%BeginExpansion
$\backslash$%
%EndExpansion
spostci%
%TCIMACRO{\TEXTsymbol{\backslash}}%
%BeginExpansion
$\backslash$%
%EndExpansion
spost\_deltaci.pdf.

Let $F\left(  x\beta\right)  $ be the estimator of interest, for example,
$F\left(  x\beta\right)  =\Pr\left(  x\beta\right)  =\Phi\left(
x\beta\right)  $ where $\Phi$ is the cumulative density function for the
standard, normal distribution. The first step is to use a Taylor expansion to
linearize the function evaluated at $\widehat{\beta}$:%
\[
F\left(  x\widehat{\beta}\right)  \approx F\left(  x\beta\right)
+(\widehat{\beta}-\beta)f(x\beta)\ ,
\]
where $f\left(  \beta\right)  =F^{\prime}\left(  \beta\right)  $ is the
derivative of $F$ evaluated at $\beta$. Then, we take the variance of both
sides of the equation:%
\[
\text{\textit{Var}}\left[  F\left(  x\widehat{\beta}\right)  \right]
\approx\text{\textit{Var}}\left[  F\left(  x\beta\right)  +(\widehat{\beta
}-\beta)f(x\beta)\right]  \ .
\]
We can easily simplify the right-hand side:%
\begin{align*}
\text{\textit{Var}}\left[  F\left(  x\beta\right)  +(\widehat{\beta}%
-\beta)f(x\beta)\right]   &  =\text{\textit{Var}}\left[  F(x\beta)\right]
+\text{\textit{Var}}\left[  (\widehat{\beta}-\beta)f(x\beta)\right] \\
&  \ \ \ +2\text{\textit{Cov}}\left[  F(x\beta),(\widehat{\beta}%
-\beta)f(x\beta)\right] \\
&  =0+\text{\textit{Var}}\left[  (\widehat{\beta}-\beta)f(x\beta)\right]  +0\\
&  =\left[  f\left(  \beta\right)  \right]  ^{2}\text{\textit{Var}}\left(
\widehat{\beta}-\beta\right) \\
&  =\left[  f(\beta)\right]  ^{2}\text{\textit{Var}}\left(  \widehat{\beta
}\right)  \ ,
\end{align*}
where we use the fact that $\beta$, $f\left(  x\beta\right)  $, and $F\left(
x\beta\right)  $ are constants.

To make our example concrete, consider binary probit where $\Pr\left(
x\widehat{\beta}\right)  =\Phi\left(  x\widehat{\beta}\right)  $ and $x$ is
any specific value. The linear expansion is:%
\begin{equation}
\Phi\left(  x\widehat{\beta}\right)  \approx\Phi\left(  x\beta\right)
+\left(  \widehat{\beta}-\beta\right)  \frac{\partial\Phi\left(
x\beta\right)  }{\partial\beta}\ , \label{probit1}%
\end{equation}
where%
\[
\frac{\partial\Phi\left(  x\beta\right)  }{\partial\beta}=x\phi\left(  \beta
x\right)  \ .
\]
Then,%
\[
\text{\textit{Var}}\left[  \Phi\left(  x\beta\right)  +\left(  \widehat{\beta
}-\beta\right)  \phi\left(  x\beta\right)  \right]  =\left[  x\phi\left(
\beta x\right)  \right]  ^{2}\text{\textit{Var}}\left(  \widehat{\beta
}\right)  \ ,
\]
which leads to the symmetric confidence interval%
\[
\left[  \Pr\left(  x\widehat{\beta}\right)  -z\sqrt{\left[  x\phi\left(
x\widehat{\beta}\right)  \right]  ^{2}\text{\textit{Var}}\left(
\widehat{\beta}\right)  }\right]  \leq\Pr\left(  x\beta\right)  \leq\left[
\Pr\left(  x\widehat{\beta}\right)  +z\sqrt{\left[  x\phi\left(
x\widehat{\beta}\right)  \right]  ^{2}\text{\textit{Var}}\left(
\widehat{\beta}\right)  }\right]  \ .
\]
Unlike the asymmetric confidence interval based on end-point transformations,
this confidence interval could include values less than $0$ or greater than
$1$.

Next, consider a discrete change $\Pr\left(  x_{a}\widehat{\beta}\right)
-\Pr\left(  x_{b}\widehat{\beta}\right)  =\Phi\left(  x_{a}\widehat{\beta
}\right)  -\Phi\left(  x_{b}\widehat{\beta}\right)  $, where $x_{a}$ and
$x_{b} $ are two values of $x$. The linearization is%
\[
\Phi\left(  x_{a}\widehat{\beta}\right)  -\Phi\left(  x_{b}\widehat{\beta
}\right)  \approx\left[  \Phi\left(  \beta x_{a}\right)  -\Phi\left(  \beta
x_{b}\right)  \right]  +\left(  \widehat{\beta}-\beta\right)  \frac
{\partial\left[  \Phi\left(  x_{a}\beta\right)  -\Phi\left(  x_{b}%
\beta\right)  \right]  }{\partial\beta}\ .
\]
Taking the variance of the right-hand-side and simplifying:%
\begin{align*}
&  \text{\textit{Var}}\left(  \left[  \Phi\left(  x_{a}\beta\right)
-\Phi\left(  x_{b}\beta\right)  \right]  +\left(  \widehat{\beta}%
-\beta\right)  \frac{\partial\left[  \Phi\left(  x_{a}\beta\right)
-\Phi\left(  x_{b}\beta\right)  \right]  }{\partial\beta}\right) \\
&  =\text{\textit{Var}}\left[  \left(  \widehat{\beta}-\beta\right)
\frac{\partial\left[  \Phi\left(  x_{a}\beta\right)  -\Phi\left(  x_{b}%
\beta\right)  \right]  }{\partial\beta}\right] \\
&  =\left(  \frac{\partial\left[  \Phi\left(  x_{a}\beta\right)  -\Phi\left(
x_{b}\beta\right)  \right]  }{\partial\beta}\right)  ^{2}\text{\textit{Var}%
}\left(  \widehat{\beta}\right) \\
&  =\left[  x_{a}^{2}\phi\left(  x_{a}\beta\right)  ^{2}+x_{b}^{2}\phi\left(
x_{b}\beta\right)  ^{2}-2x_{a}\phi\left(  x_{a}\beta\right)  x_{b}\phi\left(
x_{b}\beta\right)  \right]  \text{\textit{Var}}\left(  \widehat{\beta}\right)
\ .
\end{align*}
To evaluate it, we simply replace $\beta$ with $\widehat{\beta}$.

\subsection{Bootstrap}

The bootstrap is a computationally expensive, nonparametric technique for
making statistical inferences. The bootstrap method allows us to approximate
the variation of parameter estimates (or function of these estimates). For a
basic introduction with specific applications using Stata, we recommend Poi
(2004) and Guan (2003). For a thorough treatment of this method, see Efron and
Tibshirani (1986) and Mooney and Duval (1993). Here we present only the most
basic information.

To compute bootstrap confidence intervals for predicted outcome, the following
steps are taken:

\begin{enumerate}
\item From the original estimation sample, draw a simple random sample of size
$N$ with replacement. This is called a resample. Using the resample, estimate
the model and compute the quantities of interest.

\item Repeat step 1 $R$ times and collect the estimates from each subsample.
Use the $R$ estimates to create an empirical probability distribution of the
quantities of interest. This distribution, known as the bootstrap
distribution, is used to construct the confidence interval. Essentially, the
variation in estimates among the resamples is used to estimate the standard
error of the sample estimate.
\end{enumerate}

\noindent We use three methods for computing confidence intervals from the $R
$ empirical estimates of each parameter. The \textit{normal method} assumes
that the bootstrap distribution is approximately normal and uses the standard
deviation from the bootstrap distribution to compute the appropriate
percentiles from a normal distribution. The \textit{percentile method}
determines the $\alpha$ and $1-\alpha$ percentile from the bootstrap
distribution without any assumption about the shape of that distribution. With
this method, the bounds cannot exceed possible values for the statistic in
question. The \textit{bias-corrected method} adjusts for bias between the
predicted probabilities and the average of simulated predicted probabilities.
By default, \texttt{prvalue} uses the percentile method.

\section{Saved Results}

All results that might be useful for Monte Carlo simulations, plotting, or
other post-estimation analysis are saved to global strings or matrices. While
the typical user will not need this information, it is needed when you are
writing programs that use the results computed by \texttt{prvalue}. Additional
information can be obtained with \texttt{help \_pecollect}.

\paragraph{petype}

is a global string with the type of model being estimated. The string contains
three words. The first word contains \texttt{e(cmd)} for the model being
analyzed. Word 2 classifies the type of model as either \texttt{typical} for
all model except \texttt{zip} and \texttt{zinb} which are classified as
\texttt{twoeq}. Word 3 indicates the general type of outcome and is one of the
words: \texttt{binary}, \texttt{count}, \texttt{mlogit}, \texttt{ordered},
\texttt{regress}, or \texttt{tobit}.

\paragraph{pecimethod}

is a global string indicating the type of confidence interval that was
computed. The first word is \texttt{ml}, \texttt{delta}, \texttt{ept}, or
\texttt{bootstrap}. The second word indicates how the bootstrap confidence
intervals were computed, either \texttt{normal}, \texttt{percentile} or
\texttt{biascorrected}.

\paragraph{peinfo}

is a $3\times11$ matrix with information about the model and options used with
\texttt{prvalue}. Row 1 contains information on the current call to
\texttt{prvalue}. Row 2 contains information on \texttt{prvalue} for the model
last saved with the \texttt{save} option. Row 3 contains information on the
differences between the current and saved information. Normally, rows 1 and 2
have identical information. Columns contain the following information:

Column 1: number of right-hand-side variables.

Column 2: number of categories in the outcome.

Column 3: level for confidence interval (e.g., 95 not .95).

Column 4: $z$-value for confidence interval at given level.

Column 5: number of right-hand-side variables for inflation in \texttt{zip}
and \texttt{zinb}.

Column 6: 1 if model has no constant, else 0.

Column 7: base category for \texttt{mlogit}.

Column 8: \texttt{stdp} for binary models.

Column 9: number of requested replications for bootstrap (i.e., the number
specified by the \texttt{rep} option).

Column 10: number of completed replications for bootstrap. When a estimates
cannot be computed for a given bootstrap sample, it is not counted.

Column 11: value of the maximum number of values of predicted probabilities in
count models, corresponding to the \texttt{maxcnt} option.

\paragraph{pebase and pebase2}

contain the base values for the $x$'s in computing the predictions. The $j$th
column of \texttt{pebase} is the $j$th right-hand-side variable in the model.
The $j$th column of \texttt{pebase2} is the $j$th right-hand-side inflation
variable in \texttt{zip} or \texttt{zinb}. If \texttt{save} and \texttt{dif}
are used, the three rows are in the matrix correspond to: 1) the current
model; 2) the saved model; and 3) differences between the current and save values.

\paragraph{pepred}

contains the predictions computed by \texttt{prvalue}. This matrix has 7 rows.
I has one column for each outcome, with a minimum of 4 columns. Rows contain
the following information.

Row 1: values of the outcome category (e.g., 0, 1, 2).

Row 2: predicted probabilities for the value in row 1 for the current model.

Row 3: predictions other than probabilities for the current model.

Row 4: predicted probabilities for the value in row 1 for the saved model.

Row 5: predictions other than probabilities for the saved model.

Row 6: the difference between rows 2 and 4.

Row 7: the difference between rows 3 and 5.

\noindent For rows 2, 4 and 6, the columns contained predicted probabilities
corresponding to the categories in row 1. Rows 3, 5 and 7 contain predictions
for other quantities (where not all columns are used for all models). Column 1
contains $\mathbf{x}^{\prime}\widehat{\boldsymbol{\beta}} $ from the first
part of model. Column 2 contains $\widehat{\mu}$ for count models. Column 3:
$\mathbf{x}^{\prime}\widehat{\boldsymbol{\beta}}$ from the inflation part of
\texttt{zip} and \texttt{zinb}. Column 4: $\widehat{\Pr}\left(  \text{always
}0\right)  $ for \texttt{zip} and \texttt{zinb}.

\paragraph{peupper and pelower}

contains the lower and upper confidence limits corresponding to the
information in \texttt{pepred}.

\paragraph{peuppct, pelopct, peupbias, pelobias, peupnorm and pelonorm}

are created when the bootstrap method is used. They contain the upper and
lower limits for all three methods of computing confidence intervals:
percentile, bias-corrected, and normal approximation. Whichever method is
selected as an option in the \texttt{prvalue} command (e.g., \texttt{prvalue,
boot normal}) is also contained in \texttt{peupper} and \texttt{pelower}. The
information in each matrix corresponds to the information in \texttt{pepred}.

\section{\textbf{References}}

\begin{description}
\item Cox, Christopher and Guangqin Ma. 1995. {\textquotedblleft}Asymptotic
Confidence Bands for Generalized Nonlinear Regression
Models.{\textquotedblleft} \textit{Biometrics} 51:142-50.

\item Efron, Bradley and R. Tibshirani. 1986. {\textquotedblleft}Bootstrap
Methods for Standard Errors, Confidence Intervals, and Other Measures of
Statistical Accuracy.{\textquotedblleft} \textit{Statistical Science} 1:54-75.

\item Efron, Bradley and Robert Tibshirani. 1993. \textit{An introduction to
the bootstrap}. New York: Chapman \& Hall.

\item Guan, Weihua. 2003. From the help desk: Bootstrapped standard errors.
\textit{The Stata Journal} 3:71-80.

\item Liao, Tim Futing. 2000. {\textquotedblleft}Estimated Precision for
Predictions from Generalized Linear Models in Sociological
Research.{\textquotedblleft} \textit{Quality \& Quantity} 34:137-52.

\item Long, J. Scott. 1997. \textit{Regression Models for Categorical and
Limited Dependent Variables}. Thousand Oaks, CA: SAGE Publications.

\item Long, Scott and Jeremy Freese. Forthcoming. \textit{Regression Models
for Categorical Dependent Variables Using Stata}. Second Edition. College
Station, Texas: Stata Corporation.

\item Mooney, Christopher Z. and Robert D. Duval. 1993. \textit{Bootstrapping:
A Nonparametric Approach to Statistical Inference}. Newbury Park: SAGE Publications.

\item Poi, Brian P. 2004. From the help desk: Some bootstrapping techniques.
\textit{The Stata Journal} 4:312-328.
\end{description}

\medskip\noindent{\small File xulong-prvalue-23Aug2005.tex}


\end{document}