访问www.arxivdaily.com获取含摘要速递，涵盖CS|物理|数学|经济|统计|金融|生物|电气领域，更有搜索、收藏、发帖等功能！点击阅读原文即可访问
cs.AI人工智能，共计22篇
【1】 Heteroscedastic Temporal Variational Autoencoder For Irregularly Sampled Time Series 标题：用于不规则采样时间序列的异方差时间变分自动编码器
作者：Satya Narayan Shukla,Benjamin M. Marlin 机构：University of Massachusetts Amherst, Amherst, MA , USA 链接：https://arxiv.org/abs/2107.11350 摘要：不规则采样时间序列通常出现在多个领域，它们对标准的深度学习模型提出了重大挑战。本文提出了一种新的不规则采样时间序列概率插值的深度学习框架，称之为异方差时间变分自编码器（HeTVAE）。HeTVAE包括一个新的输入层来编码关于输入观测稀疏性的信息，一个时间VAE架构来传播由于输入稀疏性而产生的不确定性，以及一个异方差输出层来实现输出插值中的可变不确定性。我们的研究结果表明，与一系列的基线模型和传统模型，以及最近提出的使用等腰输出层的深度潜变量模型相比，所提出的结构能够更好地反映由于稀疏和不规则采样引起的随时间变化的不确定性。 摘要：Irregularly sampled time series commonly occur in several domains where they present a significant challenge to standard deep learning models. In this paper, we propose a new deep learning framework for probabilistic interpolation of irregularly sampled time series that we call the Heteroscedastic Temporal Variational Autoencoder (HeTVAE). HeTVAE includes a novel input layer to encode information about input observation sparsity, a temporal VAE architecture to propagate uncertainty due to input sparsity, and a heteroscedastic output layer to enable variable uncertainty in output interpolations. Our results show that the proposed architecture is better able to reflect variable uncertainty through time due to sparse and irregular sampling than a range of baseline and traditional models, as well as recently proposed deep latent variable models that use homoscedastic output layers.
【2】 Machine Learning with a Reject Option: A survey 标题：带拒绝选项的机器学习：综述
作者：Kilian Hendrickx,Lorenzo Perini,Dries Van der Plas,Wannes Meert,Jesse Davis 机构： BelgiumUniversity of Antwerp 链接：https://arxiv.org/abs/2107.11277 摘要：机器学习模型总是做出预测，即使它可能是不准确的。在许多决策支持应用程序中应该避免这种行为，因为错误可能会带来严重后果。尽管已经在1970年进行了研究，但带有拒绝选项的机器学习最近引起了人们的兴趣。这个机器学习子域使机器学习模型能够避免在可能出错时进行预测。这项调查的目的是提供一个概述机器学习与拒绝选项。我们介绍了导致两类拒绝的条件：歧义拒绝和新奇拒绝。此外，我们定义了现有的拒绝模型的体系结构，描述了训练此类模型的标准学习策略，并将传统的机器学习技术与拒绝联系起来。此外，我们回顾了评估模型预测和拒绝质量的策略。最后，我们提供了相关应用领域的例子，并展示了拒绝机器学习与其他机器学习研究领域的关系。 摘要：Machine learning models always make a prediction, even when it is likely to be inaccurate. This behavior should be avoided in many decision support applications, where mistakes can have severe consequences. Albeit already studied in 1970, machine learning with a reject option recently gained interest. This machine learning subfield enables machine learning models to abstain from making a prediction when likely to make a mistake. This survey aims to provide an overview on machine learning with a reject option. We introduce the conditions leading to two types of rejection, ambiguity and novelty rejection. Moreover, we define the existing architectures for models with a reject option, describe the standard learning strategies to train such models and relate traditional machine learning techniques to rejection. Additionally, we review strategies to evaluate a model's predictive and rejective quality. Finally, we provide examples of relevant application domains and show how machine learning with rejection relates to other machine learning research areas.
【3】 Adversarial Reinforced Instruction Attacker for Robust Vision-Language Navigation 标题：用于鲁棒视觉语言导航的对抗性强化指令攻击者
作者：Bingqian Lin,Yi Zhu,Yanxin Long,Xiaodan Liang,Qixiang Ye,Liang Lin 备注：None 链接：https://arxiv.org/abs/2107.11252 摘要：语言教学在自然语言导航任务中起着至关重要的作用。然而，使用有限的人工注释指令训练的导航员在不同的时间步很难准确地从复杂指令中获取关键信息，导致导航性能较差。在本文中，我们利用对抗式攻击模式，训练一个能从长指令中动态提取关键因素的更健壮的导航器。具体地说，我们提出了一种动态增强指令攻击者（DR-Attacker），它通过在不同的时间步破坏指令中最有指导意义的信息来误导导航器移动到错误的目标。通过将扰动生成过程描述为Markov决策过程，利用强化学习算法对DR攻击者进行优化，使其在导航过程中根据可学习的攻击分数依次生成扰动指令。然后，利用扰动指令作为硬样本，通过有效的对抗训练策略和辅助自监督推理任务，提高导航器的鲁棒性。在视觉与语言导航（VLN）和对话历史导航（NDH）任务上的实验结果表明，该方法优于现有的方法。此外，可视化分析显示了所提出的DR攻击者的有效性，它可以在不同的时间步成功地攻击指令中的关键信息。代码位于https://github.com/expectorlin/DR-Attacker. 摘要：Language instruction plays an essential role in the natural language grounded navigation tasks. However, navigators trained with limited human-annotated instructions may have difficulties in accurately capturing key information from the complicated instruction at different timesteps, leading to poor navigation performance. In this paper, we exploit to train a more robust navigator which is capable of dynamically extracting crucial factors from the long instruction, by using an adversarial attacking paradigm. Specifically, we propose a Dynamic Reinforced Instruction Attacker (DR-Attacker), which learns to mislead the navigator to move to the wrong target by destroying the most instructive information in instructions at different timesteps. By formulating the perturbation generation as a Markov Decision Process, DR-Attacker is optimized by the reinforcement learning algorithm to generate perturbed instructions sequentially during the navigation, according to a learnable attack score. Then, the perturbed instructions, which serve as hard samples, are used for improving the robustness of the navigator with an effective adversarial training strategy and an auxiliary self-supervised reasoning task. Experimental results on both Vision-and-Language Navigation (VLN) and Navigation from Dialog History (NDH) tasks show the superiority of our proposed method over state-of-the-art methods. Moreover, the visualization analysis shows the effectiveness of the proposed DR-Attacker, which can successfully attack crucial information in the instructions at different timesteps. Code is available at https://github.com/expectorlin/DR-Attacker.
【4】 An Improved Algorithm of Robot Path Planning in Complex Environment Based on Double DQN 标题：基于双DQN的复杂环境下机器人路径规划改进算法
作者：Fei Zhang,Chaochen Gu,Feng Yang 机构： Shanghai Jiao Tong University, Shanghai, China, Northwestern Polytechnical University, ShaanXi, China 备注：Accepted in International Conference on Guidance, Navigation and Control,2020 链接：https://arxiv.org/abs/2107.11245 摘要：实验结果表明，Deep-Q网络（DQN）在具有多种困境的环境中进行路径规划时存在一定的局限性。奖励函数可能很难建模，成功的经验转换很难在经验回放中找到。在这种背景下，本文提出了一种改进的双DQN（DDQN）算法来解决这一问题。为了实现丰富的经验回放实验，基于RRT策略重新定义了每轮训练中机器人的初始化。此外，根据A*中职位成本的定义，专门设计了免费职位奖励，以加速学习过程。仿真实验结果验证了改进的DDQN算法的有效性，机器人能够成功地学习DQN和DDQN对机器人避障和最优路径规划的影响。 摘要：Deep Q Network (DQN) has several limitations when applied in planning a path in environment with a number of dilemmas according to our experiment. The reward function may be hard to model, and successful experience transitions are difficult to find in experience replay. In this context, this paper proposes an improved Double DQN (DDQN) to solve the problem by reference to A* and Rapidly-Exploring Random Tree (RRT). In order to achieve the rich experiments in experience replay, the initialization of robot in each training round is redefined based on RRT strategy. In addition, reward for the free positions is specially designed to accelerate the learning process according to the definition of position cost in A*. The simulation experimental results validate the efficiency of the improved DDQN, and robot could successfully learn the ability of obstacle avoidance and optimal path planning in which DQN or DDQN has no effect.
【5】 Exploring Deep Registration Latent Spaces 标题：探索深度配准潜在空间
作者：Théo Estienne,Maria Vakalopoulou,Stergios Christodoulidis,Enzo Battistella,Théophraste Henry,Marvin Lerousseau,Amaury Leroy,Guillaume Chassagnon,Marie-Pierre Revel,Nikos Paragios,Eric Deutsch 机构：Deutsch, Universit´e Paris-Saclay, CentraleSup´elec, Math´ematiques et Informatique pour la, Complexit´e et les Systemes, Inria Saclay, Gif-sur-Yvette, France., Universit´e Paris-Saclay, Institut Gustave Roussy, Inserm, Radioth´erapie 备注：13 pages, 5 figures 3 figures in supplementary materials Accepted to DART 2021 workshop 链接：https://arxiv.org/abs/2107.11238 摘要：深层神经网络的可解释性是该领域最具挑战性和最有趣的问题之一。在本研究中，我们主要探讨基于深度学习的配准方法的可解释性。特别是，通过适当的模型结构和使用简单的线性投影，我们分解了编码空间，生成了一个新的基，并从经验上证明了这个基捕获了各种分解的解剖感知几何变换。我们使用两种不同的数据集进行实验，重点放在肺和海马MRI上。我们证明了这种方法可以分解正交空间中注册管道的高度卷积的潜在空间，并具有一些有趣的性质。我们希望这项工作能为更好地理解基于深度学习的注册方法提供一些启示。 摘要：Explainability of deep neural networks is one of the most challenging and interesting problems in the field. In this study, we investigate the topic focusing on the interpretability of deep learning-based registration methods. In particular, with the appropriate model architecture and using a simple linear projection, we decompose the encoding space, generating a new basis, and we empirically show that this basis captures various decomposed anatomically aware geometrical transformations. We perform experiments using two different datasets focusing on lungs and hippocampus MRI. We show that such an approach can decompose the highly convoluted latent spaces of registration pipelines in an orthogonal space with several interesting properties. We hope that this work could shed some light on a better understanding of deep learning-based registration methods.
【6】 A Fitness Landscape View on the Tuning of an Asynchronous Master-Worker EA for Nuclear Reactor Design 标题：核反应堆异步主工模拟电势调整的适宜性景观观
作者：Mathieu Muniglia,Sébastien Verel,Jean-Charles Le Pallec,Jean-Michel Do 机构： CEA (french Commissariat a l’Energie Atomique), France, Universit´e du Littoral Cˆote d’Opale, LISIC, France 备注：None 链接：https://arxiv.org/abs/2107.11201 摘要：在间歇式可再生能源应用的背景下，我们建议对核电站控制棒的主要变量进行优化，以提高其负荷跟踪能力。设计问题是一个基于多物理模拟器的具有代价评估的黑盒组合优化问题。因此，我们使用了一种并行异步主从进化算法，可以扩展到上千个计算单元。一个主要问题是算法参数的调整。对这一代价昂贵的现实问题进行了适应度景观分析，表明可以根据适应度景观特征的低成本估计来调整变异参数。 摘要：In the context of the introduction of intermittent renewable energies, we propose to optimize the main variables of the control rods of a nuclear power plant to improve its capability to load-follow. The design problem is a black-box combinatorial optimization problem with expensive evaluation based on a multi-physics simulator. Therefore, we use a parallel asynchronous master-worker Evolutionary Algorithm scaling up to thousand computing units. One main issue is the tuning of the algorithm parameters. A fitness landscape analysis is conducted on this expensive real-world problem to show that it would be possible to tune the mutation parameters according to the low-cost estimation of the fitness landscape features.
【7】 Bias Loss for Mobile Neural Networks 标题：移动神经网络的偏置损耗
作者：Lusine Abrahamyan,Valentin Ziatchin,Yiming Chen,Nikos Deligiannis 机构：Vrije Universiteit Brussel, Brussels, Belgium, PicsArt Inc., San Francisco, USA 备注：Accepted at ICCV2021 链接：https://arxiv.org/abs/2107.11170 摘要：近年来，紧凑卷积神经网络（CNNs）在性能上有了显著的提高。然而，在参数众多的情况下，它们仍然不能提供与CNNs相同的预测能力。这些层所捕捉到的多样甚至丰富的特征是这些成功的cnn的一个重要特征。然而，大型cnn与紧凑型cnn在这一特性上的差异很少被研究。在紧凑型CNNs中，由于参数数目有限，不可能获得丰富的特征，而特征多样性成为CNNs的一个重要特征。在模型推理期间从数据点导出的激活映射中存在的不同特征可以指示存在一组区分不同类的对象所必需的唯一描述符。相比之下，具有低特征多样性的数据点可能无法提供足够数量的唯一描述符来进行有效预测；我们称之为随机预测。随机预测会对优化过程产生负面影响，并损害最终性能。本文提出通过重塑标准交叉熵来解决随机预测问题，使其偏向于具有有限数量独特描述特征的数据点。我们的新的偏差损失集中在一组有价值的数据点的训练，并防止了大量的样本与不良的学习特性误导优化过程。此外，为了说明多样性的重要性，我们提出了一系列SkipNet模型，这些模型的体系结构增加了最后一层中唯一描述符的数量。我们的Skipnet-M比MobileNetV3-Large的分类精度高1%。 摘要：Compact convolutional neural networks (CNNs) have witnessed exceptional improvements in performance in recent years. However, they still fail to provide the same predictive power as CNNs with a large number of parameters. The diverse and even abundant features captured by the layers is an important characteristic of these successful CNNs. However, differences in this characteristic between large CNNs and their compact counterparts have rarely been investigated. In compact CNNs, due to the limited number of parameters, abundant features are unlikely to be obtained, and feature diversity becomes an essential characteristic. Diverse features present in the activation maps derived from a data point during model inference may indicate the presence of a set of unique descriptors necessary to distinguish between objects of different classes. In contrast, data points with low feature diversity may not provide a sufficient amount of unique descriptors to make a valid prediction; we refer to them as random predictions. Random predictions can negatively impact the optimization process and harm the final performance. This paper proposes addressing the problem raised by random predictions by reshaping the standard cross-entropy to make it biased toward data points with a limited number of unique descriptive features. Our novel Bias Loss focuses the training on a set of valuable data points and prevents the vast number of samples with poor learning features from misleading the optimization process. Furthermore, to show the importance of diversity, we present a family of SkipNet models whose architectures are brought to boost the number of unique descriptors in the last layers. Our Skipnet-M can achieve 1% higher classification accuracy than MobileNetV3 Large.
【8】 Constellation: Learning relational abstractions over objects for compositional imagination 标题：星座：学习对象上的关系抽象以进行构图想象
作者：James C. R. Whittington,Rishabh Kabra,Loic Matthey,Christopher P. Burgess,Alexander Lerchner 机构： Factorisedsensory representations are easily re-combined to represent 1UniversityofOxford 2WorkdoneatDeepMind 3DeepMind 4Wayve 链接：https://arxiv.org/abs/2107.11153 摘要：学习视觉场景的结构化表示是目前连接感知和推理的主要瓶颈。虽然基于狭缝的模型已经取得了令人兴奋的进展，它可以学习将场景分割成多组对象，但是学习整个对象组的配置特性仍在探索中。为了解决这个问题，我们引入了Constellation，一个学习静态视觉场景的关系抽象的网络，并将这些抽象概括为感官的特殊性，从而为抽象关系推理提供了一个潜在的基础。我们进一步证明，这个基础，连同语言联想，提供了一种以新的方式想象感官内容的方法。这项工作是在视觉关系的显式表示和复杂的认知过程中使用它们的第一步。 摘要：Learning structured representations of visual scenes is currently a major bottleneck to bridging perception with reasoning. While there has been exciting progress with slot-based models, which learn to segment scenes into sets of objects, learning configurational properties of entire groups of objects is still under-explored. To address this problem, we introduce Constellation, a network that learns relational abstractions of static visual scenes, and generalises these abstractions over sensory particularities, thus offering a potential basis for abstract relational reasoning. We further show that this basis, along with language association, provides a means to imagine sensory content in new ways. This work is a first step in the explicit representation of visual relationships and using them for complex cognitive procedures.
【9】 User Preferences and the Shortest Path 标题：用户首选项和最短路径
作者：Isabella Kreller,Bernd Ludwig 机构：University of Regensburg, Universit¨atsstraße , D-, Regensburg, Germany 链接：https://arxiv.org/abs/2107.11150 摘要：室内导航系统利用最短路径算法来计算路线。为了定义“最短路径”，必须根据应用领域的理论和启发式方法来指定成本函数。对于室内布线领域，我们调查的理论和标准，在文献中确定为必要的人体路径规划。我们驱动定量定义，并将其集成到一个成本函数中，分别对每个标准进行加权。然后，我们应用穷尽的网格搜索来找到导致理想成本函数的权重。”这里的“理想”定义为引导算法规划与人类选择的路线最为相似的路线。为了探讨在改进的寻径算法中应该考虑哪些准则，在过去的研究中，考虑了11种对路径选择有有利影响的因素。每个因素分别包含在Dijkstra算法中，并由此计算出与雷根斯堡大学学生选择的实际路线相似的路线的相似性。这允许对这些因素的影响进行定量评估，并进一步构成了一种直接比较它们的方法。减少转弯、街道、旋转门、入口、电梯的数量以及上述因素的组合被发现具有积极的影响，并生成比最短路径更受欢迎的路径。结果证明，转变和标准的结合是最有影响的。 摘要：Indoor navigation systems leverage shortest path algorithms to calculate routes. In order to define the "shortest path", a cost function has to be specified based on theories and heuristics in the application domain. For the domain of indoor routing, we survey theories and criteria identified in the literature as essential for human path planning. We drive quantitative definitions and integrate them into a cost function that weights each of the criteria separately. We then apply an exhaustive grid search to find weights that lead to an ideal cost function. "Ideal" here is defined as guiding the algorithm to plan routes that are most similar to those chosen by humans. To explore which criteria should be taken into account in an improved pathfinding algorithm, eleven different factors whose favorable impact on route selection has been established in past research were considered. Each factor was included separately in the Dijkstra algorithm and the similarity of thus calculated routes to the actual routes chosen by students at the University of Regensburg was determined. This allows for a quantitative assessment of the factors' impact and further constitutes a way to directly compare them. A reduction of the number of turns, streets, revolving doors, entryways, elevators as well as the combination of the aforementioned factors was found to have a positive effect and generate paths that were favored over the shortest path. Turns and the combination of criteria turned out to be most impactful.
【10】 Malware Analysis with Artificial Intelligence and a Particular Attention on Results Interpretability 标题：基于人工智能的恶意软件分析及其结果可解释性
作者：Benjamin Marais,Tony Quertier,Christophe Chesneau 链接：https://arxiv.org/abs/2107.11100 摘要：恶意软件检测与分析是近年来网络安全领域的一个研究热点。事实上，模糊处理技术的发展，例如打包，需要特别注意检测恶意软件的最新变种。通常的检测方法不一定提供解释结果的工具。因此，本文提出了一种基于二值文件到灰度图像转换的模型，其准确率达到88%。此外，所提出的模型能够以85%的精度确定样本是否被打包或加密。它使我们能够分析结果并采取适当的行动。此外，通过在检测模型上应用注意机制，我们有可能识别文件中哪些部分看起来可疑。这种工具对于数据分析人员应该非常有用，它弥补了常见检测模型的可解释性不足，并且有助于理解为什么一些恶意文件未被检测到。 摘要：Malware detection and analysis are active research subjects in cybersecurity over the last years. Indeed, the development of obfuscation techniques, as packing, for example, requires special attention to detect recent variants of malware. The usual detection methods do not necessarily provide tools to interpret the results. Therefore, we propose a model based on the transformation of binary files into grayscale image, which achieves an accuracy rate of 88%. Furthermore, the proposed model can determine if a sample is packed or encrypted with a precision of 85%. It allows us to analyze results and act appropriately. Also, by applying attention mechanisms on detection models, we have the possibility to identify which part of the files looks suspicious. This kind of tool should be very useful for data analysts, it compensates for the lack of interpretability of the common detection models, and it can help to understand why some malicious files are undetected.
【11】 Generative adversarial networks in time series: A survey and taxonomy 标题：时间序列中的生成性对抗网络：综述与分类学
作者：Eoin Brophy,Zhengwei Wang,Qi She,Tomas Ward 机构：Infant Research Centre & School of Computing, Dublin City University, Ireland, ByteDance AI Lab, China, Tomás Ward, Insight SFI Research Centre for Data Analytics 链接：https://arxiv.org/abs/2107.11098 摘要：在过去的几年中，生成性对抗网络的研究呈指数增长。它们的影响主要体现在计算机视觉领域，真实感图像和视频处理，特别是生成，取得了重大进展。虽然这些计算机视觉的进步已经引起了人们的广泛关注，但GAN的应用已经在时间序列和序列生成等学科中多样化。作为GANs的一个相对较新的利基，实地调查正在进行中，以开发高质量、多样化和私有的时间序列数据。在本文中，我们回顾了GAN变量设计的时间序列相关的应用。我们提出了离散变量和连续变量的分类，其中变量处理离散时间序列和连续时间序列数据。在这里我们展示这个领域最新最流行的文学作品；它们的体系结构、结果和应用程序。我们还提供了最流行的评估指标及其在应用程序中的适用性列表。此外，还讨论了这些机构的隐私措施以及处理敏感数据的进一步保护和方向。我们的目标是明确和简洁的框架在这一领域的最新和最先进的研究及其应用到现实世界的技术。 摘要：Generative adversarial networks (GANs) studies have grown exponentially in the past few years. Their impact has been seen mainly in the computer vision field with realistic image and video manipulation, especially generation, making significant advancements. While these computer vision advances have garnered much attention, GAN applications have diversified across disciplines such as time series and sequence generation. As a relatively new niche for GANs, fieldwork is ongoing to develop high quality, diverse and private time series data. In this paper, we review GAN variants designed for time series related applications. We propose a taxonomy of discrete-variant GANs and continuous-variant GANs, in which GANs deal with discrete time series and continuous time series data. Here we showcase the latest and most popular literature in this field; their architectures, results, and applications. We also provide a list of the most popular evaluation metrics and their suitability across applications. Also presented is a discussion of privacy measures for these GANs and further protections and directions for dealing with sensitive data. We aim to frame clearly and concisely the latest and state-of-the-art research in this area and their applications to real-world technologies.
【12】 HURRA! Human readable router anomaly detection 标题：哈拉！人类可读的路由器异常检测
作者：Jose M. Navarro,Dario Rossi 备注：None 链接：https://arxiv.org/abs/2107.11078 摘要：本文介绍了HURRA系统，该系统旨在减少操作人员在网络故障排除过程中所花费的时间。为此，它包括两个模块，在任何异常检测算法之后插入：（i）第一个注意机制，根据当前特征与异常的关系对其进行排序；（ii）第二个模块能够无缝地结合先前的专家知识，而不需要任何人的交互或决策。我们在一组真实的路由器数据集上展示了这些简单过程的有效性，这些数据集来自数十个isp，表现出丰富的异常和非常异构的kpi集，在这些数据集上，我们通过解决疑难解答问题的操作员手动收集注释的地面真相。我们的实验评估表明：（i）所提出的系统在与专家达成高度一致方面是有效的，（ii）即使是简单的统计方法也能够从过去案例中获得的专家知识中提取有用的信息，以进一步提高性能，最后（iii）实时部署的主要困难涉及异常检测算法的自动选择及其超参数的调整。 摘要：This paper presents HURRA, a system that aims to reduce the time spent by human operators in the process of network troubleshooting. To do so, it comprises two modules that are plugged after any anomaly detection algorithm: (i) a first attention mechanism, that ranks the present features in terms of their relation with the anomaly and (ii) a second module able to incorporates previous expert knowledge seamlessly, without any need of human interaction nor decisions. We show the efficacy of these simple processes on a collection of real router datasets obtained from tens of ISPs which exhibit a rich variety of anomalies and very heterogeneous set of KPIs, on which we gather manually annotated ground truth by the operator solving the troubleshooting ticket. Our experimental evaluation shows that (i) the proposed system is effective in achieving high levels of agreement with the expert, that (ii) even a simple statistical approach is able to extracting useful information from expert knowledge gained in past cases to further improve performance and finally that (iii) the main difficulty in live deployment concerns the automated selection of the anomaly detection algorithm and the tuning of its hyper-parameters.
【13】 LocalGLMnet: interpretable deep learning for tabular data 标题：LocalGLMnet：表格数据的可解释深度学习
作者：Ronald Richman,Mario V. Wüthrich 机构：Mario V. W¨uthrich† 链接：https://arxiv.org/abs/2107.11059 摘要：深度学习模型在统计建模中得到了广泛的应用，因为它们导致了非常有竞争力的回归模型，通常比经典的统计模型（如广义线性模型）表现更好。深度学习模型的缺点是其解很难解释和解释，变量选择也不容易，因为深度学习模型在内部以不透明的方式解决特征工程和变量选择问题。受广义线性模型吸引人的结构启发，我们提出了一种新的网络结构，该结构与广义线性模型具有相似的特性，但得益于表示学习的艺术，它提供了优越的预测能力。这种新的体系结构允许表格数据的变量选择和校准的深度学习模型的解释，事实上，我们的方法提供了一种基于Shapley值和综合梯度的加法分解。 摘要：Deep learning models have gained great popularity in statistical modeling because they lead to very competitive regression models, often outperforming classical statistical models such as generalized linear models. The disadvantage of deep learning models is that their solutions are difficult to interpret and explain, and variable selection is not easily possible because deep learning models solve feature engineering and variable selection internally in a nontransparent way. Inspired by the appealing structure of generalized linear models, we propose a new network architecture that shares similar features as generalized linear models, but provides superior predictive power benefiting from the art of representation learning. This new architecture allows for variable selection of tabular data and for interpretation of the calibrated deep learning model, in fact, our approach provides an additive decomposition in the spirit of Shapley values and integrated gradients.
【14】 MCDAL: Maximum Classifier Discrepancy for Active Learning 标题：MCDAL：主动学习的最大分类器差异
作者：Jae Won Cho,Dong-Jin Kim,Yunjae Jung,In So Kweon 机构：KAIST, South Korea. 备注：10 pages 链接：https://arxiv.org/abs/2107.11049 摘要：目前最先进的主动学习方法大多利用生成性对抗网络（GAN）进行样本获取；然而，GAN通常具有不稳定性和对超参数的敏感性。与这些方法相比，本文提出了一种新的主动学习框架，称为最大分类器差异主动学习（MCDAL），它考虑了多个分类器之间的预测差异。特别地，我们利用两个辅助分类层，通过最大化它们之间的差异来学习更紧密的决策边界。直观地说，辅助分类层预测结果的差异反映了预测结果的不确定性。在这方面，我们提出了一种新的方法来利用分类器的差异来实现主动学习。我们还提供了一个解释，我们的想法与现有的基于GAN的主动学习方法和领域适应框架。此外，我们还通过实验证明了我们的方法的实用性，在主动学习环境下，我们的方法在一些图像分类和语义分割数据集上的性能超过了最先进的方法。 摘要：Recent state-of-the-art active learning methods have mostly leveraged Generative Adversarial Networks (GAN) for sample acquisition; however, GAN is usually known to suffer from instability and sensitivity to hyper-parameters. In contrast to these methods, we propose in this paper a novel active learning framework that we call Maximum Classifier Discrepancy for Active Learning (MCDAL) which takes the prediction discrepancies between multiple classifiers. In particular, we utilize two auxiliary classification layers that learn tighter decision boundaries by maximizing the discrepancies among them. Intuitively, the discrepancies in the auxiliary classification layers' predictions indicate the uncertainty in the prediction. In this regard, we propose a novel method to leverage the classifier discrepancies for the acquisition function for active learning. We also provide an interpretation of our idea in relation to existing GAN based active learning methods and domain adaptation frameworks. Moreover, we empirically demonstrate the utility of our approach where the performance of our approach exceeds the state-of-the-art methods on several image classification and semantic segmentation datasets in active learning setups.
【15】 Pruning Ternary Quantization 标题：修剪三值量化
作者：Dan Liu,Xi Chen,Jie Fu,Xue Liu 机构：McGill University, Chen Xi, MILA 链接：https://arxiv.org/abs/2107.10998 摘要：提出了一种简单有效的对称三值量化方法——剪枝三值量化法。该方法将神经网络权值显著压缩为[-1,0,1]的稀疏三值，从而减少计算、存储和内存占用。我们证明了PTQ可以通过简单的剪枝和L2投影将规则权值转换为三值正交基。此外，我们引入一个精化的直通估计器来确定和稳定量化的权值。我们的方法在ResNet-18结构上可以提供最多46倍的压缩比，可以接受的准确率为65.36%，优于主流方法。此外，PTQ可以将ResNet-18模型从46mb压缩到955KB（~48x），ResNet-50模型从99mb压缩到3.3MB（~30x），而ImageNet上的top-1精度分别从69.7%下降到65.3%和76.15%下降到74.47%。我们的方法统一了剪枝和量化，从而提供了一个范围的大小精度权衡。 摘要：We propose pruning ternary quantization (PTQ), a simple, yet effective, symmetric ternary quantization method. The method significantly compresses neural network weights to a sparse ternary of [-1,0,1] and thus reduces computational, storage, and memory footprints. We show that PTQ can convert regular weights to ternary orthonormal bases by simply using pruning and L2 projection. In addition, we introduce a refined straight-through estimator to finalize and stabilize the quantized weights. Our method can provide at most 46x compression ratio on the ResNet-18 structure, with an acceptable accuracy of 65.36%, outperforming leading methods. Furthermore, PTQ can compress a ResNet-18 model from 46 MB to 955KB (~48x) and a ResNet-50 model from 99 MB to 3.3MB (~30x), while the top-1 accuracy on ImageNet drops slightly from 69.7% to 65.3% and from 76.15% to 74.47%, respectively. Our method unifies pruning and quantization and thus provides a range of size-accuracy trade-off.
【16】 Resource Efficient Mountainous Skyline Extraction using Shallow Learning 标题：基于浅层学习的资源高效山地天际线提取
作者：Touqeer Ahmad,Ebrahim Emami,Martin Čadík,George Bebis 机构：∗Vision and Security Technology Lab, University of Colorado at Colorado Springs, USA, †Department of Computer Science and Engineering, University of Nevada, Reno, USA, Brno University of Technology, Czech Republic 备注：Accepted at International Joint Conference on Neural Networks, 2021 链接：https://arxiv.org/abs/2107.10997 摘要：天际线在山地视觉地理定位、行星探测器/无人机定位/导航以及虚拟/增强现实应用中起着关键作用。本文提出了一种新的山区天际线检测方法，采用浅层学习方法学习一组滤波器来区分天山边界和其他来自不同区域的边缘。与以前的方法不同，我们的方法要么依赖于显式特征描述子的提取及其分类，要么依赖于精细调整一般场景解析深度网络进行天空分割，我们的方法学习基于局部结构分析的线性滤波器。在测试时，对于每一个候选边缘像素，根据像素的结构张量从学习的滤波器集中选择一个滤波器，然后应用于其周围的面片。然后利用动态规划求解得到的多级图的最短路径问题，得到天山边界。所提出的方法在提供可比性能的同时，计算速度比以前的方法快，并且更适合于资源受限的平台，例如移动设备、行星漫游者和无人机。我们比较了我们提出的方法与早期的天际线检测方法使用四种不同的数据集。我们的代码可从url获得{https://github.com/TouqeerAhmad/skyline_detection}. 摘要：Skyline plays a pivotal role in mountainous visual geo-localization and localization/navigation of planetary rovers/UAVs and virtual/augmented reality applications. We present a novel mountainous skyline detection approach where we adapt a shallow learning approach to learn a set of filters to discriminate between edges belonging to sky-mountain boundary and others coming from different regions. Unlike earlier approaches, which either rely on extraction of explicit feature descriptors and their classification, or fine-tuning general scene parsing deep networks for sky segmentation, our approach learns linear filters based on local structure analysis. At test time, for every candidate edge pixel, a single filter is chosen from the set of learned filters based on pixel's structure tensor, and then applied to the patch around it. We then employ dynamic programming to solve the shortest path problem for the resultant multistage graph to get the sky-mountain boundary. The proposed approach is computationally faster than earlier methods while providing comparable performance and is more suitable for resource constrained platforms e.g., mobile devices, planetary rovers and UAVs. We compare our proposed approach against earlier skyline detection methods using four different data sets. Our code is available at url{https://github.com/TouqeerAhmad/skyline_detection}.
【17】 Learning Quadruped Locomotion Policies with Reward Machines 标题：用奖励机学习四足行走策略
作者：David DeFazio,Shiqi Zhang 机构：Binghamton University 链接：https://arxiv.org/abs/2107.10969 摘要：腿部机器人已经被证明在非结构化环境中是有效的。虽然四足机器人的运动策略学习已经取得了很大的成功，但是如何结合人类的知识来促进这一学习过程的研究却很少。在本文中，我们证明了人类知识的形式LTL公式可以应用于四足动物的运动学习奖励机（RM）的框架。仿真实验结果表明，基于RM的方法可以方便地定义不同的运动风格，并有效地学习所定义风格的运动策略。 摘要：Legged robots have been shown to be effective in navigating unstructured environments. Although there has been much success in learning locomotion policies for quadruped robots, there is little research on how to incorporate human knowledge to facilitate this learning process. In this paper, we demonstrate that human knowledge in the form of LTL formulas can be applied to quadruped locomotion learning within a Reward Machine (RM) framework. Experimental results in simulation show that our RM-based approach enables easily defining diverse locomotion styles, and efficiently learning locomotion policies of the defined styles.
【18】 Domain Generalization under Conditional and Label Shifts via Variational Bayesian Inference 标题：基于变分贝叶斯推理的条件移位和标签移位下的区域泛化
作者：Xiaofeng Liu,Bo Hu,Linghao Jin,Xu Han,Fangxu Xing,Jinsong Ouyang,Jun Lu,Georges EL Fakhri,Jonghye Woo 机构：Dept. of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, USA, Beth Israel Deaconess Medical Center and Harvard Medical School, Boston, MA, USA, National University of Singapore, Singapore 备注：30th International Joint Conference on Artificial Intelligence (IJCAI) 2021 链接：https://arxiv.org/abs/2107.10931 摘要：在这项工作中，我们提出了一种领域泛化（DG）方法来学习多个标记的源领域，并将知识转移到训练中无法访问的目标领域。考虑到固有的条件转移和标签转移，我们希望对齐$p（x | y）$和$p（y）$。然而，广泛使用的领域不变特征学习（IFL）方法依赖于对齐边缘概念移位w.r.t.$p（x）$，这是基于一个不切实际的假设，$p（y）$是跨领域不变的。在此基础上，我们提出了一种新的变分贝叶斯推理框架，通过潜在空间中的先验分布匹配来实现条件分布对齐w.r.t.$p（x | y）$，同时考虑了后验对齐时的边缘标签移位w.r.t.$p（y）$。在各种基准上的大量实验表明，我们的框架对标签移动具有鲁棒性，跨域精度得到显著提高，从而实现了优于传统IFL框架的性能。 摘要：In this work, we propose a domain generalization (DG) approach to learn on several labeled source domains and transfer knowledge to a target domain that is inaccessible in training. Considering the inherent conditional and label shifts, we would expect the alignment of $p(x|y)$ and $p(y)$. However, the widely used domain invariant feature learning (IFL) methods relies on aligning the marginal concept shift w.r.t. $p(x)$, which rests on an unrealistic assumption that $p(y)$ is invariant across domains. We thereby propose a novel variational Bayesian inference framework to enforce the conditional distribution alignment w.r.t. $p(x|y)$ via the prior distribution matching in a latent space, which also takes the marginal label shift w.r.t. $p(y)$ into consideration with the posterior alignment. Extensive experiments on various benchmarks demonstrate that our framework is robust to the label shift and the cross-domain accuracy is significantly improved, thereby achieving superior performance over the conventional IFL counterparts.
【19】 On the Certified Robustness for Ensemble Models and Beyond 标题：关于系综模型及以后的认证稳健性
作者：Zhuolin Yang,Linyi Li,Xiaojun Xu,Bhavya Kailkhura,Tao Xie,Bo Li 机构： University of Illinois at Urbana-Champaign, USA, Lawrence Livermore National Laboratory, USA 备注：57 pages, 11 pages for main text 链接：https://arxiv.org/abs/2107.10873 摘要：最近的研究表明，深层神经网络（DNN）容易受到对抗性例子的攻击，这些例子的目的是通过增加小幅度的扰动来误导DNN。为了防御这种攻击，对于一个ML模型，经验和理论防御方法都得到了广泛的研究。在这项工作中，我们的目的是分析和提供集成ML模型的鲁棒性，以及不同集成协议鲁棒性的充分必要条件。尽管整体模型在经验上比单一模型更具稳健性；令人惊讶的是，我们发现，在认证的稳健性方面，标准集成模型与单一模型相比只取得了微小的改进。因此，为了探讨保证提供可证明鲁棒集成ML模型的条件，我们首先证明了在模型光滑性假设下，多样性梯度和大置信区间是可证明鲁棒集成ML模型的充要条件。在此基础上，给出了基于该集成的前平滑策略的有界模型平滑度分析。我们还证明了在温和的条件下，集成模型总是比单基模型具有更高的鲁棒性。受这些理论发现的启发，我们提出了轻量级多样性正则化训练（DRT）来训练可证明鲁棒的集成ML模型。大量的实验表明，我们的DRT增强的集成比现有的单一和集成ML模型具有更高的认证鲁棒性，证明了MNIST、CIFAR-10和ImageNet数据集上最先进的认证L2鲁棒性。 摘要：Recent studies show that deep neural networks (DNN) are vulnerable to adversarial examples, which aim to mislead DNNs by adding perturbations with small magnitude. To defend against such attacks, both empirical and theoretical defense approaches have been extensively studied for a single ML model. In this work, we aim to analyze and provide the certified robustness for ensemble ML models, together with the sufficient and necessary conditions of robustness for different ensemble protocols. Although ensemble models are shown more robust than a single model empirically; surprisingly, we find that in terms of the certified robustness the standard ensemble models only achieve marginal improvement compared to a single model. Thus, to explore the conditions that guarantee to provide certifiably robust ensemble ML models, we first prove that diversified gradient and large confidence margin are sufficient and necessary conditions for certifiably robust ensemble models under the model-smoothness assumption. We then provide the bounded model-smoothness analysis based on the proposed Ensemble-before-Smoothing strategy. We also prove that an ensemble model can always achieve higher certified robustness than a single base model under mild conditions. Inspired by the theoretical findings, we propose the lightweight Diversity Regularized Training (DRT) to train certifiably robust ensemble ML models. Extensive experiments show that our DRT enhanced ensembles can consistently achieve higher certified robustness than existing single and ensemble ML models, demonstrating the state-of-the-art certified L2-robustness on MNIST, CIFAR-10, and ImageNet datasets.
【20】 Joint Shapley values: a measure of joint feature importance 标题：关节Shapley值：关节特征重要性的度量
作者：Chris Harris,Richard Pymar,Colin Rowat 机构：Visual Alpha, Tokyo, Japan, Economics, Mathematics and Statistics, Birkbeck College University of London, UK, University of Birmingham, UK 备注：Source code available at this https URL 链接：https://arxiv.org/abs/2107.11357 摘要：Shapley值是可解释人工智能中最广泛使用的特征重要性的模型不可知度量之一：它有明确的公理基础，保证唯一存在，并且作为特征对模型预测的平均影响有明确的解释。我们引入了联合Shapley值，它直接扩展了Shapley公理。这保留了经典的Shapley值的直觉：联合Shapley值度量一组特征对模型预测的平均影响。证明了联合Shapley值的唯一性。游戏结果表明，联合Shapley值与现有的交互指数不同，后者评估了一组特征中一个特征的效果。由此导出ML属性问题中的联合Shapley值，我们就可以第一次度量特征集对模型预测的联合影响。在具有二进制特征的数据集中，我们提出了一种保留效率特性的全局值计算方法。 摘要：The Shapley value is one of the most widely used model-agnostic measures of feature importance in explainable AI: it has clear axiomatic foundations, is guaranteed to uniquely exist, and has a clear interpretation as a feature's average effect on a model's prediction. We introduce joint Shapley values, which directly extend the Shapley axioms. This preserves the classic Shapley value's intuitions: joint Shapley values measure a set of features' average effect on a model's prediction. We prove the uniqueness of joint Shapley values, for any order of explanation. Results for games show that joint Shapley values present different insights from existing interaction indices, which assess the effect of a feature within a set of features. Deriving joint Shapley values in ML attribution problems thus gives us the first measure of the joint effect of sets of features on model predictions. In a dataset with binary features, we present a presence-adjusted method for calculating global values that retains the efficiency property.
【21】 Generating Large-scale Dynamic Optimization Problem Instances Using the Generalized Moving Peaks Benchmark 标题：使用广义移动峰值基准生成大规模动态优化问题实例
作者：Mohammad Nabi Omidvar,Danial Yazdani,Juergen Branke,Xiaodong Li,Shengxiang Yang,Xin Yao 机构：School of Computing, University of Leeds, and Leeds University Business School, Leeds, United Kingdom., Guangdong Provincial Key Laboratory of Brain-inspired Intelligent Computation, Department of Computer Science and 备注：arXiv admin note: text overlap with arXiv:2106.06174 链接：https://arxiv.org/abs/2107.11019 摘要：本文描述了广义移动峰值基准（GMPB）及其如何用于生成连续大规模动态优化问题的问题实例。它提出了一套15个基准问题，相关的源代码，和一个性能指标，旨在进行比较研究和大型动态优化比赛。尽管它的主要目的是为竞赛提供一个连贯的基础，但它的通用性允许感兴趣的读者使用本文档作为指导，设计定制的问题实例，以调查超出所提供的基准套件范围的问题。为此，我们解释了GMPB的模块化结构，以及如何将其组成部分组合起来，形成具有各种可控特征的问题实例，这些可控特征包括从单峰到高度多峰、从对称到高度不对称、从平滑到高度不规则以及各种程度的可变交互作用和病态。 摘要：This document describes the generalized moving peaks benchmark (GMPB) and how it can be used to generate problem instances for continuous large-scale dynamic optimization problems. It presents a set of 15 benchmark problems, the relevant source code, and a performance indicator, designed for comparative studies and competitions in large-scale dynamic optimization. Although its primary purpose is to provide a coherent basis for running competitions, its generality allows the interested reader to use this document as a guide to design customized problem instances to investigate issues beyond the scope of the presented benchmark suite. To this end, we explain the modular structure of the GMPB and how its constituents can be assembled to form problem instances with a variety of controllable characteristics ranging from unimodal to highly multimodal, symmetric to highly asymmetric, smooth to highly irregular, and various degrees of variable interaction and ill-conditioning.
【22】 A reinforcement learning approach to resource allocation in genomic selection 标题：基因组选择中资源分配的强化学习方法
作者：Saba Moeinizade,Guiping Hu,Lizhi Wang 机构：Industrial and Manufacturing Systems Engineering Department, Iowa State University 备注：18 pages,5 figures 链接：https://arxiv.org/abs/2107.10901 摘要：基因组选择（GS）是植物育种家用来选择个体进行交配并产生新一代物种的技术。资源配置是全球战略的关键因素。在每一个选择周期中，育种家都面临着预算分配的选择，以进行杂交并产生下一代的育种亲本。受人工智能问题强化学习最新进展的启发，我们开发了一种基于强化学习的算法来自动学习在不同世代的育种中分配有限的资源。我们在马尔可夫决策过程（MDP）的框架下，通过定义状态空间和动作空间，对问题进行了数学描述。为了避免状态空间的爆炸，提出了一个整数线性规划来量化资源和时间之间的权衡。最后，我们提出一个价值函数近似方法来估计行动价值函数，然后发展贪婪策略改进技术来寻找最佳资源。通过一个实际数据的例子，我们证明了该方法在提高遗传增益方面的有效性。 摘要：Genomic selection (GS) is a technique that plant breeders use to select individuals to mate and produce new generations of species. Allocation of resources is a key factor in GS. At each selection cycle, breeders are facing the choice of budget allocation to make crosses and produce the next generation of breeding parents. Inspired by recent advances in reinforcement learning for AI problems, we develop a reinforcement learning-based algorithm to automatically learn to allocate limited resources across different generations of breeding. We mathematically formulate the problem in the framework of Markov Decision Process (MDP) by defining state and action spaces. To avoid the explosion of the state space, an integer linear program is proposed that quantifies the trade-off between resources and time. Finally, we propose a value function approximation method to estimate the action-value function and then develop a greedy policy improvement technique to find the optimal resources. We demonstrate the effectiveness of the proposed method in enhancing genetic gain using a case study with realistic data.
