统计学中的优化以及R实现—马学俊

统计学中的优化以及R实现马学俊　统计学中的优化以及R实现献给我的家人、恩师和所有在学术道路上帮助我的人♣RD2蓦然回首谈起优化，统计人无人不知，无人不晓。在经典的统计学中，我们通常力求得到一个“完美”的解。这个解我们当时认为完美无瑕，然而当面对繁杂的世界时，我们依然心惊胆战。因为，我们不知道这样的“完美”会持续多久。刚接触统计，记得那是最小二乘的天下，或者其兄弟加权最小二乘的天下。后来被告知，天下原本不是上述两个兄弟，是优化大帝的，犹如醍醐灌顶，梦中惊醒。遥想原来与君有过一面之缘（研究生必修课）只是未钟情；而今喜相逢，一壶浊酒，笑看秋月春风。缘分呀！说来惭愧，对于大帝不知，犹如有眼不识泰山。谈起优化，其繁体字是優化。看来不是好糊弄之辈，此人“心”在中间，单“人”靠边站，犹如包拯之无私，李逵之耿直。于是，不能再溜须拍马敷衍了事了！吾只能双耳不闻窗外事，上下而求索了。原打算写一本“老少”皆宜的读物。但优化大帝犹如道，神龙见首不见尾。对于我这种刚修行之人，优化大帝近在咫尺却遥不可及呀。只能怀着忐忑的心情，记录一些拜访优化大帝的感受。书名原定《统计学中的优化之他说》。之所以取名《他说》，是因为这些都是作者的一些“道听途说”。如有不妥，请大家不吝赐教，贵在参与。如果问题，麻烦联系鄙人([e-mail])，感激涕零。马学俊2016年1月于言蹊堂3统计学中的优化以及R实现前言随着统计学的发展，统计计算越来越重要，尤其是大数据时代。优化是统计计算的重要组成部分。很多统计方法或模型的实现都依赖它，如分位数回归、复合分位数回归、广义线性回归和众数回归等。R软件是一个非常强大的统计软件。由于其灵活性和易操作性，它目前几乎超越了SAS和SPSS等统计软件。它不仅仅是统计方法实现的软件，更是学习统计的有利工具。您可以通过“?命令”找到研究的参考文献，如安装quantreg，然后运行?rq；您可以查看分位数回归研究的文献；您可以通过包，从而学习新的统计方法；您可以查看统计模型或方法实现的R包摘要。如总结了R中的优化包；您还可以通过使用的技巧等。本书主要讲解优化在R中实现，阐述优化在统计学中的应用。书的素材有些来自于R包的帮组文档，有些来自于作者研究中遇到的问题。全书主要解决以下优化问题(暂定，因为内容在更新)：∙线性规划:min𝑥A𝑥𝑠.𝑡.B𝑥≤𝑤其中A∈R𝑛×𝑝，𝑥∈R𝑝，B∈R𝑚×𝑝和𝑤∈R𝑚。∙二次规划:min𝑥12𝑥⊤D𝑥−𝑑⊤𝑥𝑠.𝑡.A⊤𝑥≥𝑏其中D∈R𝑝×𝑝，𝑥∈R𝑝，A∈R𝑝×𝑛和𝑏∈R𝑛。∙非线性方程：𝑓(𝑥)=0其中𝑓:R𝑝→R𝑝，𝑥∈R𝑝。4∙非线性优化：min𝑥𝑓(𝑥)𝑠.𝑡.𝑥𝑙≤𝑥≤𝑥𝑢,𝑏𝑙≤A𝑥≤𝑏𝑢,𝑐𝑙≤𝑐(𝑥)≤𝑐𝑢其中𝑥∈R𝑝，𝑓(𝑥)：R𝑝→R，A∈R𝑛×𝑝。𝑥𝑙和𝑥𝑢是参数𝑥的上下界，𝑏𝑙和𝑏𝑢是𝑥的线性组合A𝑥的上下界（线性约束），𝑐𝑙和𝑐𝑢是𝑥的非线性约束𝑐(𝑥)的上下界。本书是学习统计中的优化及R实现的入门教程，可以作为统计学专业的课外读物。限于作者水平，不妥或谬误之处在所难免，恳请大家赐教。非常感谢！马学俊2016年2月于言蹊堂5统计学中的优化以及R实现缩写与记号∙s.t.：约束∙𝑌：向量∙X：矩阵（不包含向量）∙R𝑘：𝑘维欧式空间∙⊤：转置6目目目录录录目录····································································7第1章线性规划························································11.1一般表达式·····················································11.2R的实现························································11.3应用····························································11.3.1分位数回归··············································21.3.2复合分位数回归·········································51.4参考文献·······················································9第2章二次规划························································112.1一般表达式·····················································112.2R的实现························································112.3另一种表达式···················································112.4应用····························································122.4.1约束最小二乘············································122.5参考文献·······················································13第3章非线性方程·····················································153.1一般表达式·····················································153.2R的实现························································153.2.1引言·····················································163.2.2SANE和DF-SANE之局部收敛···························173.2.3SANE和DF-SANE之全局收敛···························183.2.4SANE和DF-SANE在BB中的实现·························193.2.5BBsolve的选择··········································197统计学中的优化以及R实现3.2.6multiStart的使用········································223.3应用····························································243.3.1带常数项的泊松回归·····································243.4参考文献·······················································27第4章参数约束的非线性优化··········································294.1一般表达式·····················································294.2R的实现························································294.3应用····························································304.3.1众数回归················································304.4参考文献·······················································37第5章线性和非线性约束的非线性优化·································395.1一般表达式·····················································395.2R的实现························································405.2.1初始值··················································405.2.2目标函数和梯度·········································405.2.3约束·····················································415.2.4线性约束················································415.2.5非线性约束和其梯度·····································425.2.6数值梯度················································425.2.7一些设置················································435.2.8输出结果················································435.3另一种表达式···················································435.4应用····························································445.4.1复杂的例子··············································445.4.2经验似然筛选方法·······································495.5参考文献·······················································548第第第1章章章线线线性性性规规规划划划线性规划(LinearProgramming)是最常见的优化方法。统计学中很多问题都可以转化为线性规划，如分位数回归、复合分位数回归等。本章安排如下：1.1给出线性规划的一般表达式；1.2介绍R的软件实现；1.3阐述线性规划在统计学中的应用。1.1一一一般般般表表表达达达式式式线性规划的一般表达式是min𝑥A𝑥𝑠.𝑡.B𝑥≤𝑤其中A∈R𝑛×𝑝，𝑥∈R𝑝，B∈R𝑚×𝑝和𝑤∈R𝑚。≤可以是≥。一般来说，我们需要将统计问题转化为线性规划的一般表达式。1.2R的的的实实实现现现lpSolve包(Berkelaar等2015)中lp函数可以实现线性规划，其一般形式是lp(direction=”min”,objective.in,const.mat,const.dir,const.rhs)用法：objective.in:Aconst.mat：Bconst.dir：条件的符号（”¡,””¡=,””=