人工智能学术速递[7.15]

访问www.arxivdaily.com获取含摘要速递，涵盖CS|物理|数学|经济|统计|金融|生物|电气领域，更有搜索、收藏、发帖等功能！点击阅读原文即可访问

cs.AI人工智能，共计54篇

【1】 DULA: A Differentiable Ergonomics Model for Postural Optimization in Physical HRI 标题：Dula：用于体能HRI体位优化的可微人机工程学模型

作者：Amir Yazdani,Roya Sabbagh Novin,Andrew Merryweather,Tucker Hermans 机构：∗University of Utah Robotics Center, Salt Lake City, UT, †NVIDIA, Seattle, WA 链接：https://arxiv.org/abs/2107.06875 摘要：人机工程学和人体舒适性是物理人机交互应用中的重要问题。定义一个准确且易于使用的人体工程学评估模型是提供姿势校正反馈以改善操作员健康和舒适度的一个重要步骤。为了实现有效的计算，先前提出的自动工效学评估和校正工具对工效学家在实践中使用的金标准评估工具进行了近似或简化。为了保持评估的质量，同时提高计算的考虑，我们引入了杜拉，一个可微的和连续的人体工程学模型学习复制流行的和科学验证的鲁拉评估。我们表明，杜拉提供的评估相当于鲁拉，同时提供计算效益。我们强调杜拉的实力，在演示梯度为基础的姿势优化模拟遥操作任务。 摘要：Ergonomics and human comfort are essential concerns in physical human-robot interaction applications. Defining an accurate and easy-to-use ergonomic assessment model stands as an important step in providing feedback for postural correction to improve operator health and comfort. In order to enable efficient computation, previously proposed automated ergonomic assessment and correction tools make approximations or simplifications to gold-standard assessment tools used by ergonomists in practice. In order to retain assessment quality, while improving computational considerations, we introduce DULA, a differentiable and continuous ergonomics model learned to replicate the popular and scientifically validated RULA assessment. We show that DULA provides assessment comparable to RULA while providing computational benefits. We highlight DULA's strength in a demonstration of gradient-based postural optimization for a simulated teleoperation task.

【2】 Generalisation in Neural Networks Does not Require Feature Overlap 标题：神经网络中的泛化不需要特征重叠

作者：Jeff Mitchell,Jeffrey S. Bowers 机构：School of Computing, Edinburgh Napier University, Colinton Road, Edinburgh, UK., School of Psychological Science, University of, Bristol, Priory Road, Bristol, UK. 备注：19 pages, 3 Figures. Submitted to Cognition 链接：https://arxiv.org/abs/2107.06872 摘要：训练数据和测试数据之间的共享特征是人工神经网络中推广所必需的，这是这些模型的支持者和批评者的共同假设。在这里，我们展示了卷积结构通过将它们应用于两个众所周知的挑战来避免这一限制，这两个挑战是基于学习身份函数和控制单词序列的学习规则。在每一种情况下，测试集上的成功表现都需要概括训练数据中不存在的特征，这对于标准连接模型来说通常是不可行的。然而，我们的实验证明，当神经网络结合卷积结构所采用的权值共享时，它们可以成功地解决这类问题。在图像处理领域，这样的架构旨在反映这样的图像所描绘的自然世界在空间平移下的对称性。我们讨论对称性在这两项任务中的作用及其与概括的联系。 摘要：That shared features between train and test data are required for generalisation in artificial neural networks has been a common assumption of both proponents and critics of these models. Here, we show that convolutional architectures avoid this limitation by applying them to two well known challenges, based on learning the identity function and learning rules governing sequences of words. In each case, successful performance on the test set requires generalising to features that were not present in the training data, which is typically not feasible for standard connectionist models. However, our experiments demonstrate that neural networks can succeed on such problems when they incorporate the weight sharing employed by convolutional architectures. In the image processing domain, such architectures are intended to reflect the symmetry under spatial translations of the natural world that such images depict. We discuss the role of symmetry in the two tasks and its connection to generalisation.

【3】 Reinforced Hybrid Genetic Algorithm for the Traveling Salesman Problem 标题：求解旅行商问题的增强型混合遗传算法

作者：Jiongzhi Zheng,Menglei Chen,Jialun Zhong,Kun He 机构：School of Computer Science and Technology, Huazhong University of Science and Technology, China, Institute of Artificial Intelligence 链接：https://arxiv.org/abs/2107.06870 摘要：针对著名的NP难旅行商问题（TSP），提出了一种强大的增强型混合遗传算法（RHGA）。RHGA将强化学习技术与著名的边缘装配交叉遗传算法（EAX-GA）和Lin-Kernighan-Helsgaun（LKH）局部搜索启发式算法相结合。在提出的混合机制的帮助下，EAX-GA的遗传进化和LKH的局部搜索可以相互促进。基于Q学习的强化学习技术进一步促进了混合遗传算法的发展。在138个著名和广泛使用的TSP基准上的实验结果表明，该方法具有良好的性能。 摘要：We propose a powerful Reinforced Hybrid Genetic Algorithm (RHGA) for the famous NP-hard Traveling Salesman Problem (TSP). RHGA combines reinforcement learning technique with the well-known Edge Assembly Crossover genetic algorithm (EAX-GA) and the Lin-Kernighan-Helsgaun (LKH) local search heuristic. With the help of the proposed hybrid mechanism, the genetic evolution of EAX-GA and the local search of LKH can boost each other's performance. And the reinforcement learning technique based on Q-learning further promotes the hybrid genetic algorithm. Experimental results on 138 well-known and widely used TSP benchmarks, with the number of cities ranging from 1,000 to 85,900, demonstrate the excellent performance of the proposed method.

【4】 Differentiable Programming of Reaction-Diffusion Patterns 标题：反应扩散模式的可微规划

作者：Alexander Mordvintsev,Ettore Randazzo,Eyvind Niklasson 机构：Google 备注：ALIFE 2021 链接：https://arxiv.org/abs/2107.06862 摘要：反应扩散（RD）系统提供了一个计算框架，控制着自然界中许多模式的形成过程。目前的研发系统设计实践归结为试错参数搜索。我们提出了一种可微优化方法来学习RD系统参数，以便在二维平面上进行基于实例的纹理合成。为此，我们将RD系统表示为神经细胞自动机的变体，并使用特定于任务的可微损失函数。通过我们的方法生成的RD系统表现出健壮的、非平凡的“类生命”行为。 摘要：Reaction-Diffusion (RD) systems provide a computational framework that governs many pattern formation processes in nature. Current RD system design practices boil down to trial-and-error parameter search. We propose a differentiable optimization method for learning the RD system parameters to perform example-based texture synthesis on a 2D plane. We do this by representing the RD system as a variant of Neural Cellular Automata and using task-specific differentiable loss functions. RD systems generated by our method exhibit robust, non-trivial 'life-like' behavior.

【5】 Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Pot 标题：基于熔池的多智能体强化学习可伸缩性评估

作者：Joel Z. Leibo,Edgar Duéñez-Guzmán,Alexander Sasha Vezhnevets,John P. Agapiou,Peter Sunehag,Raphael Koster,Jayd Matyas,Charles Beattie,Igor Mordatch,Thore Graepel 备注：None 链接：https://arxiv.org/abs/2107.06857 摘要：现有的多智能体强化学习（MARL）评估套件并没有将新情境的泛化作为其主要目标（不同于监督学习基准）。我们的贡献，熔炉，是一个泥灰评估套件，填补了这一空白，并使用强化学习，以减少所需的人力劳动，创造新的测试场景。这是因为一个代理的行为构成了另一个代理环境的一部分。为了展示可伸缩性，我们创建了80多个独特的测试场景，涵盖了广泛的研究主题，如社会困境、互惠、资源共享和任务划分。我们将这些测试场景应用到标准的MARL训练算法中，并演示了meltpot如何揭示仅从训练性能来看并不明显的弱点。 摘要：Existing evaluation suites for multi-agent reinforcement learning (MARL) do not assess generalization to novel situations as their primary objective (unlike supervised-learning benchmarks). Our contribution, Melting Pot, is a MARL evaluation suite that fills this gap, and uses reinforcement learning to reduce the human labor required to create novel test scenarios. This works because one agent's behavior constitutes (part of) another agent's environment. To demonstrate scalability, we have created over 80 unique test scenarios covering a broad range of research topics such as social dilemmas, reciprocity, resource sharing, and task partitioning. We apply these test scenarios to standard MARL training algorithms, and demonstrate how Melting Pot reveals weaknesses not apparent from training performance alone.

【6】 Extreme Precipitation Seasonal Forecast Using a Transformer Neural Network 标题：基于Transformer神经网络的极端降水季节预报

作者：Daniel Salles Civitarese,Daniela Szwarcman,Bianca Zadrozny,Campbell Watson 链接：https://arxiv.org/abs/2107.06846 摘要：气候变化的一个影响是极端降水事件的频率和强度增加。然而，自信地预测季节尺度极端降水的可能性仍然是一个突出的挑战。在这里，我们提出了一种方法来预测最大日降水量的分位数在每个星期长达六个月前使用时间融合Transformer（TFT）模式。通过在两个地区的实验，我们比较了TFT预测与两个基线的预测：气候学和校准的ECMWF SEAS5集合预测（S5）。我们的结果表明，在6个月的提前期分位数风险方面，TFT预测显著优于S5预测，并且与气候学相比显示出总体上的小改进。TFT也对气候所不能的偏离正常值作出积极反应。 摘要：An impact of climate change is the increase in frequency and intensity of extreme precipitation events. However, confidently predicting the likelihood of extreme precipitation at seasonal scales remains an outstanding challenge. Here, we present an approach to forecasting the quantiles of the maximum daily precipitation in each week up to six months ahead using the temporal fusion transformer (TFT) model. Through experiments in two regions, we compare TFT predictions with those of two baselines: climatology and a calibrated ECMWF SEAS5 ensemble forecast (S5). Our results show that, in terms of quantile risk at six month lead time, the TFT predictions significantly outperform those from S5 and show an overall small improvement compared to climatology. The TFT also responds positively to departures from normal that climatology cannot.

【7】 Mixing Human Demonstrations with Self-Exploration in Experience Replay for Deep Reinforcement Learning 标题：深度强化学习中经验回放中人的演示与自我探索的融合

作者：Dylan Klein,Akansel Cosgun 机构：Monash University, Australia 备注：2 pages. Submitted to ICDL 2021 Workshop on Human aligned Reinforcement Learning for Autonomous Agents and Robots 链接：https://arxiv.org/abs/2107.06840 摘要：我们研究了在回放缓冲区中使用人类演示数据进行深度强化学习的效果。我们使用了一种策略梯度方法和一个修改的经验回放缓冲区，其中一个人的示范经验是以给定的概率抽样。我们分析了在任务中使用演示数据的不同比率，其中一个代理试图达到一个目标，同时避免障碍。我们的结果表明，当纯自我探索和纯示范训练的个体有相似的成功率时，纯示范模型以较少的步骤收敛到解的速度更快。 摘要：We investigate the effect of using human demonstration data in the replay buffer for Deep Reinforcement Learning. We use a policy gradient method with a modified experience replay buffer where a human demonstration experience is sampled with a given probability. We analyze different ratios of using demonstration data in a task where an agent attempts to reach a goal while avoiding obstacles. Our results suggest that while the agents trained by pure self-exploration and pure demonstration had similar success rates, the pure demonstration model converged faster to solutions with less number of steps.

【8】 A Review on Edge Analytics: Issues, Challenges, Opportunities, Promises, Future Directions, and Applications 标题：边缘分析：问题、挑战、机遇、前景、未来方向与应用

作者：Sabuzima Nayak,Ripon Patgiri,Lilapati Waikhom,Arif Ahmed 机构：National Institute of Technology Silchar, India, Ericsson, Sweden 备注：Submitted to Elsevier for possible publication 链接：https://arxiv.org/abs/2107.06835 摘要：边缘技术旨在将云资源（特别是计算、存储和网络）带到边缘设备的附近，即生产和消费数据的智能设备。在边缘设备中嵌入计算和应用导致边缘技术中出现了两个新概念，即边缘计算和边缘分析。Edge analytics使用一些技术或算法来分析Edge设备生成的数据。随着边缘分析技术的出现，边缘设备已经成为一个完整的集合。目前，Edge analytics无法为分析技术的执行提供全面支持。边缘设备不能执行先进的和复杂的分析算法后，各种限制，如有限的电源，内存大小，有限的资源等。本文旨在提供一个详细的讨论边缘分析。一个明确的解释，以区分边缘技术的三个概念，即边缘设备，边缘计算和边缘分析，以及他们的问题。此外，本文还讨论了边缘分析的实现，以解决零售、农业、工业和医疗保健等各个领域的许多问题。此外，本文还对最新边缘分析的研究论文进行了严格的回顾，探讨了存在的问题、面临的挑战、研究机遇及其发展方向和应用。 摘要：Edge technology aims to bring Cloud resources (specifically, the compute, storage, and network) to the closed proximity of the Edge devices, i.e., smart devices where the data are produced and consumed. Embedding computing and application in Edge devices lead to emerging of two new concepts in Edge technology, namely, Edge computing and Edge analytics. Edge analytics uses some techniques or algorithms to analyze the data generated by the Edge devices. With the emerging of Edge analytics, the Edge devices have become a complete set. Currently, Edge analytics is unable to provide full support for the execution of the analytic techniques. The Edge devices cannot execute advanced and sophisticated analytic algorithms following various constraints such as limited power supply, small memory size, limited resources, etc. This article aims to provide a detailed discussion on Edge analytics. A clear explanation to distinguish between the three concepts of Edge technology, namely, Edge devices, Edge computing, and Edge analytics, along with their issues. Furthermore, the article discusses the implementation of Edge analytics to solve many problems in various areas such as retail, agriculture, industry, and healthcare. In addition, the research papers of the state-of-the-art edge analytics are rigorously reviewed in this article to explore the existing issues, emerging challenges, research opportunities and their directions, and applications.

【9】 A Review-based Taxonomy for Secure Health Care Monitoring: Wireless Smart Cameras 标题：基于回顾的安全医疗监控分类法：无线智能摄像机

作者：Ravi Teja Batchu,Abeer Alsadoon,P. W. C. Prasad,Rasha S. Ali,Tarik A. Rashid,Ghossoon Alsadoon,Oday D. Jerew 机构：(,). A Review-based Taxonomy for Secure Health Care Monitoring: Wireless Smart Cameras, Journal of Applied Security Research 备注：None 链接：https://arxiv.org/abs/2107.06833 摘要：健康档案数据安全是电子健康系统面临的主要挑战之一。身份验证是支持存储数据机密性、完整性和可用性的基本安全服务之一。这项研究的重点是在医疗保健部门的病人和医疗记录的安全存储数据安全和未经授权的访问是一个持续的问题。一个潜在的解决方案来自生物特征识别，尽管它们的使用可能很耗时，并且会减慢数据检索的速度。这项研究旨在克服这些挑战，并通过添加指纹形式的生物特征来加强医疗保健部门的数据访问控制。提出的模型应用于医疗领域，包括采集、网络通信和使用生物特征的身份验证（CNA），它取代了现有的基于密码的访问控制方法。传感器然后收集数据，并通过使用网络（无线或Zig-bee），建立连接后，连接分析和数据管理工作，处理和汇总的数据。随后，向应用程序的经过身份验证的用户授予访问权限。这种基于物联网的生物认证系统有助于有效识别，并确保患者、记录和其他敏感数据的机密性、完整性和可靠性。提出的解决方案提供了对医疗数据的可靠访问，并通过用户和设备身份验证过程实现了安全访问。该模型通过对医疗保健系统中的用户进行身份验证来实现对数据的访问控制，以减少数据的篡改或窃取。 摘要：Health records data security is one of the main challenges in e-health systems. Authentication is one of the essential security services to support the stored data confidentiality, integrity, and availability. This research focuses on the secure storage of patient and medical records in the healthcare sector where data security and unauthorized access is an ongoing issue. A potential solution comes from biometrics, although their use may be time-consuming and can slow down data retrieval. This research aims to overcome these challenges and enhance data access control in the healthcare sector through the addition of biometrics in the form of fingerprints. The proposed model for application in the healthcare sector consists of Collection, Network communication, and Authentication (CNA) using biometrics, which replaces an existing password-based access control method. A sensor then collects data and by using a network (wireless or Zig-bee), a connection is established, after connectivity analytics and data management work which processes and aggregate the data. Subsequently, access is granted to authenticated users of the application. This IoT-based biometric authentication system facilitates effective recognition and ensures confidentiality, integrity, and reliability of patients, records and other sensitive data. The proposed solution provides reliable access to healthcare data and enables secure access through the process of user and device authentication. The proposed model has been developed for access control to data through the authentication of users in healthcare to reduce data manipulation or theft.

【10】 Efficient Set of Vectors Search 标题：有效的向量搜索集

作者：Michael Leybovich,Oded Shmueli 机构：Technion, Haifa, Israel 备注：6 pages, 0 figures 链接：https://arxiv.org/abs/2107.06817 摘要：我们考虑两个集合之间的相似性度量$$$和$b$向量，平衡向量对之间的平均余弦距离和最大余弦距离，一个来自集合$$$，另一个来自集合$b$。作为这项措施的动机，我们提出了一个数据库中的血统跟踪。为了实际实现这一度量，我们需要一种近似搜索算法，在给定一组向量$a$和一组向量$B_1，…，B_n$的情况下，该算法能快速定位出使相似度最大化的集合$B_i$。对于所有集合都是单例集合（本质上每个集合都是单个向量）的情况，存在已知的有效近似搜索算法，例如，树搜索算法的近似版本、局部敏感散列（LSH）、向量量化（VQ）和邻近图算法。在这项工作中，我们提出了一般情况下的近似搜索算法。这些算法的基本思想是通过一个“长”的单个向量对一组向量进行编码。 摘要：We consider a similarity measure between two sets $A$ and $B$ of vectors, that balances the average and maximum cosine distance between pairs of vectors, one from set $A$ and one from set $B$. As a motivation for this measure, we present lineage tracking in a database. To practically realize this measure, we need an approximate search algorithm that given a set of vectors $A$ and sets of vectors $B_1,...,B_n$, the algorithm quickly locates the set $B_i$ that maximizes the similarity measure. For the case where all sets are singleton sets, essentially each is a single vector, there are known efficient approximate search algorithms, e.g., approximated versions of tree search algorithms, locality-sensitive hashing (LSH), vector quantization (VQ) and proximity graph algorithms. In this work, we present approximate search algorithms for the general case. The underlying idea in these algorithms is encoding a set of vectors via a "long" single vector.

【11】 Synthesis in Style: Semantic Segmentation of Historical Documents using Synthetic Data 标题：风格合成：利用合成数据对历史文献进行语义切分

作者：Christian Bartz,Hendrik Rätz,Haojin Yang,Joseph Bethge,Christoph Meinel 机构：Hasso Plattner Institute, University of Potsdam, Prof.-Dr.-Helmert Str. ,-, Potsdam, Germany 备注：Code available at: this https URL 链接：https://arxiv.org/abs/2107.06777 摘要：在历史文档的自动分析中，最紧迫的问题之一是带注释的训练数据的可用性。本文提出了一种新的文档图像语义分割训练数据合成方法。我们利用在StyleGAN生成器的中间特征中发现的簇来同时合成RGB和标签图像。我们的模型可以应用于任何扫描文档的数据集，而无需对单个图像进行手动注释，因为每个模型都是根据数据集定制的。在我们的实验中，我们证明在我们的合成数据上训练的模型可以在开放的基准数据集上达到有竞争力的性能。 摘要：One of the most pressing problems in the automated analysis of historical documents is the availability of annotated training data. In this paper, we propose a novel method for the synthesis of training data for semantic segmentation of document images. We utilize clusters found in intermediate features of a StyleGAN generator for the synthesis of RGB and label images at the same time. Our model can be applied to any dataset of scanned documents without the need for manual annotation of individual images, as each model is custom-fit to the dataset. In our experiments, we show that models trained on our synthetic data can reach competitive performance on open benchmark datasets for line segmentation.

【12】 Fast and Slow Enigmas and Parental Guidance 标题：快慢谜与父母指导

作者：Zarathustra Goertzel,Karel Chvalovský,Jan Jakubův,Miroslav Olšák,Josef Urban 机构： Czech Technical University in Prague, Prague, Czech Republic, University of Innsbruck, Austria 备注：23 pages, 11 tables, 1 figure, submitted to FroCoS 2021 链接：https://arxiv.org/abs/2107.06750 摘要：我们描述了几个添加到ENIGMA系统中，用于指导E自动定理证明器中的子句选择。首先，我们通过添加基于服务器的GPU评估显著加快了它的神经引导速度。第二个添加是由快速基于权重的抑制滤波器驱动的，这些滤波器目前在E和Prover9等系统中使用。这样的系统可以通过训练ENIGMA的快速版本来实现更智能的预过滤，从而变得更智能。这导致了可训练的快速和慢速思维的结合，这比仅快速和仅慢速的方法都有所改进。第三种附加语是基于“父母判断孩子”，即在推理产生从句之前可能拒绝推理。这是由标准的进化机制驱动的，在当前的种群中，产生所有可能的后代总是有成本的。这通过不使用更昂贵的方法评估所有子句来节省时间，并提供生成子句的补充视图。这些方法在一个来自Mizar数学库的大型基准上进行了评估，显示出比最新技术有很好的改进。 摘要：We describe several additions to the ENIGMA system that guides clause selection in the E automated theorem prover. First, we significantly speed up its neural guidance by adding server-based GPU evaluation. The second addition is motivated by fast weight-based rejection filters that are currently used in systems like E and Prover9. Such systems can be made more intelligent by instead training fast versions of ENIGMA that implement more intelligent pre-filtering. This results in combinations of trainable fast and slow thinking that improves over both the fast-only and slow-only methods. The third addition is based on "judging the children by their parents", i.e., possibly rejecting an inference before it produces a clause. This is motivated by standard evolutionary mechanisms, where there is always a cost to producing all possible offsprings in the current population. This saves time by not evaluating all clauses by more expensive methods and provides a complementary view of the generated clauses. The methods are evaluated on a large benchmark coming from the Mizar Mathematical Library, showing good improvements over the state of the art.

【13】 Artificial Intelligence in PET: an Industry Perspective 标题：PET中的人工智能：产业视角

作者：Arkadiusz Sitek,Sangtae Ahn,Evren Asma,Adam Chandler,Alvin Ihsani,Sven Prevrhal,Arman Rahmim,Babak Saboury,Kris Thielemans 机构：Healthcare, America, Columbia, Provincial Medical Imaging Physicist, BC Cancer, Canada, Sciences, Clinical Center, National Institutes of Health, Department of Computer Science 链接：https://arxiv.org/abs/2107.06747 摘要：人工智能（AI）在积极影响和推进医学成像，包括正电子发射断层扫描（PET）成像应用方面具有巨大潜力。AI有能力增强和优化PET成像链的各个方面，从患者安排、患者设置、协议、数据采集、探测器信号处理、重建、图像处理和解释。人工智能带来了特定行业的挑战，需要解决和克服这些挑战，以最大限度地发挥人工智能在PET中的未来潜力。本文概述了人工智能的发展、标准化、商业化和临床应用所面临的行业挑战，并探讨了人工智能在不久的将来对PET成像的潜在增强。特别是，按需图像重建、人工智能和定制设计的数据处理工作流程的结合可能为创新带来新的可能性，这将对行业和最终患者产生积极影响。 摘要：Artificial intelligence (AI) has significant potential to positively impact and advance medical imaging, including positron emission tomography (PET) imaging applications. AI has the ability to enhance and optimize all aspects of the PET imaging chain from patient scheduling, patient setup, protocoling, data acquisition, detector signal processing, reconstruction, image processing and interpretation. AI poses industry-specific challenges which will need to be addressed and overcome to maximize the future potentials of AI in PET. This paper provides an overview of these industry-specific challenges for the development, standardization, commercialization, and clinical adoption of AI, and explores the potential enhancements to PET imaging brought on by AI in the near future. In particular, the combination of on-demand image reconstruction, AI, and custom designed data processing workflows may open new possibilities for innovation which would positively impact the industry and ultimately patients.

【14】 MDE4QAI: Towards Model-Driven Engineering for Quantum Artificial Intelligence 标题：MDE4QAI：走向量子人工智能的模型驱动工程

作者：Armin Moin,Moharram Challenger,Atta Badii,Stephan Günnemann 机构：Dept. of Informatics, Technical University of Munich, Germany, Dept. of Computer Science, University of Antwerp, & Flanders Make, Belgium, University of Reading, United Kingdom, Stephan G¨unnemann 备注：Preliminary Version - Vision Paper 链接：https://arxiv.org/abs/2107.06708 摘要：在过去的十年里，人工智能（AI）为软件系统提供了巨大的新的可能性和机遇，同时也提出了新的需求和要求。特别是，机器学习（ML）在几乎所有垂直应用领域都被证明是有用的。尽管人工智能的其他子学科，如智能代理和多代理系统（MAS）并没有得到同样程度的推广，但它们仍然有潜力融入主流技术栈和生态系统，例如，由于物联网（IoT）和智能网络物理系统（CPS）的不断普及。然而，在未来的十年里，一个从经典计算到量子计算（QC）的前所未有的范式转变有望出现，也许会出现一个量子经典混合模型。我们期望模型驱动工程（MDE）范式在量子和量子经典混合应用中成为一个推动者和促进者，因为它已经证明在物联网、智能CPS和人工智能等高度复杂的领域中是有益的，具有固有的异构软硬件平台和api。这不仅包括自动代码生成，还包括自动模型检查和验证，以及早期设计阶段的模型分析，以及设计时和运行时的模型到模型转换。本文主要研究量子人工智能中的MDE，以及一种综合上述各方面的整体方法。 摘要：Over the past decade, Artificial Intelligence (AI) has provided enormous new possibilities and opportunities, but also new demands and requirements for software systems. In particular, Machine Learning (ML) has proven useful in almost every vertical application domain. Although other sub-disciplines of AI, such as intelligent agents and Multi-Agent Systems (MAS) did not become promoted to the same extent, they still possess the potential to be integrated into the mainstream technology stacks and ecosystems, for example, due to the ongoing prevalence of the Internet of Things (IoT) and smart Cyber-Physical Systems (CPS). However, in the decade ahead, an unprecedented paradigm shift from classical computing towards Quantum Computing (QC) is expected, with perhaps a quantum-classical hybrid model. We expect the Model-Driven Engineering (MDE) paradigm to be an enabler and a facilitator, when it comes to the quantum and the quantum-classical hybrid applications as it has already proven beneficial in the highly complex domains of IoT, smart CPS and AI with inherently heterogeneous hardware and software platforms, and APIs. This includes not only automated code generation, but also automated model checking and verification, as well as model analysis in the early design phases, and model-to-model transformations both at the design-time and at the runtime. In this paper, the vision is focused on MDE for Quantum AI, and a holistic approach integrating all of the above.

【15】 Deep Adaptive Multi-Intention Inverse Reinforcement Learning 标题：深度自适应多意图逆强化学习

作者：Ariyan Bighashdel,Panagiotis Meletis,Pavol Jancura,Gijs Dubbelman 机构：Eindhoven University of Technology, AZ Eindhoven, The Netherlands 备注：Accepted for presentation at ECML/PKDD 2021 链接：https://arxiv.org/abs/2107.06692 摘要：提出了一种深度逆强化学习（IRL）框架，该框架可以从未标记的专家演示中学习先验未知数量的非线性奖励函数。为此，我们利用Dirichlet过程中的工具，提出了一种同时考虑复杂和未知数量奖励函数的自适应方法。利用条件最大熵原理，将专家的多意图行为建模为潜在意图分布的混合模型，并推导了两种算法来估计深度奖励网络的参数以及来自未标记演示的专家意图数。在三个基准上对所提出的算法进行了评估，其中两个已在本研究中特别扩展到多意图IRL，并与已知的基准进行了比较。我们通过几个实验证明了我们的算法相对于现有方法的优势以及在线推断的好处，而不是预先确定专家意图的数量。 摘要：This paper presents a deep Inverse Reinforcement Learning (IRL) framework that can learn an a priori unknown number of nonlinear reward functions from unlabeled experts' demonstrations. For this purpose, we employ the tools from Dirichlet processes and propose an adaptive approach to simultaneously account for both complex and unknown number of reward functions. Using the conditional maximum entropy principle, we model the experts' multi-intention behaviors as a mixture of latent intention distributions and derive two algorithms to estimate the parameters of the deep reward network along with the number of experts' intentions from unlabeled demonstrations. The proposed algorithms are evaluated on three benchmarks, two of which have been specifically extended in this study for multi-intention IRL, and compared with well-known baselines. We demonstrate through several experiments the advantages of our algorithms over the existing approaches and the benefits of online inferring, rather than fixing beforehand, the number of expert's intentions.

【16】 Safer Reinforcement Learning through Transferable Instinct Networks 标题：基于可转移直觉网络的更安全强化学习

作者：Djordje Grbic,Sebastian Risi 机构：IT University of Copenhagen, Copenhagen 备注：The paper was accepted in the ALIFE 2021 conference 链接：https://arxiv.org/abs/2107.06686 摘要：随机探索是强化学习（RL）发现有效策略的主要机制之一。然而，当在安全关键环境中在线学习时，它可能会导致不良或灾难性的结果。事实上，安全学习是实现在部署过程中可以学习的真实代理的主要障碍之一。确保代理遵守硬限制的一种方法是显式配置它们可以操作的边界。虽然这在某些情况下可能奏效，但我们并不总是掌握明确的先验信息，即哪些国家和行动可能导致危险的接近危险的国家。在这里，我们提出了一种方法，其中附加策略可以覆盖主策略，并提供更安全的替代操作。在我们的本能调节RL（IR^2L）方法中，一个“本能”网络被训练来识别不受欢迎的情况，同时保护学习策略不进入它们。本能网络是在一个任务上预先训练的，在这个任务中，犯错是安全的，然后转移到安全地学习新任务是至关重要的环境中。我们在OpenAI安全健身房领域演示了IR^2L，在达到类似任务性能的同时，它在训练期间收到的安全违规数量明显低于基线RL方法。 摘要：Random exploration is one of the main mechanisms through which reinforcement learning (RL) finds well-performing policies. However, it can lead to undesirable or catastrophic outcomes when learning online in safety-critical environments. In fact, safe learning is one of the major obstacles towards real-world agents that can learn during deployment. One way of ensuring that agents respect hard limitations is to explicitly configure boundaries in which they can operate. While this might work in some cases, we do not always have clear a-priori information which states and actions can lead dangerously close to hazardous states. Here, we present an approach where an additional policy can override the main policy and offer a safer alternative action. In our instinct-regulated RL (IR^2L) approach, an "instinctual" network is trained to recognize undesirable situations, while guarding the learning policy against entering them. The instinct network is pre-trained on a single task where it is safe to make mistakes, and transferred to environments in which learning a new task safely is critical. We demonstrate IR^2L in the OpenAI Safety gym domain, in which it receives a significantly lower number of safety violations during training than a baseline RL approach while reaching similar task performance.

【17】 Improved SAT models for NFA learning 标题：用于NFA学习的改进SAT模型

作者：Frédéric Lardeux,Eric Monfroy 机构： University of Angers 备注：None 链接：https://arxiv.org/abs/2107.06672 摘要：语法推理是研究从单词中学习自动机和语法的算法。我们专注于从单词样本中学习大小为k的不确定有限自动机。为此，我们将问题描述为SAT模型。由于生成的SAT实例非常庞大，我们提出了一些模型改进，包括变量数量、子句数量和子句大小。这些改进显著地减少了实例，但是以更长的生成时间为代价。因此，我们试图平衡实例大小与生成和求解时间之间的关系。我们还进行了一些实验比较，并分析了我们的各种模型改进。 摘要：Grammatical inference is concerned with the study of algorithms for learning automata and grammars from words. We focus on learning Nondeterministic Finite Automaton of size k from samples of words. To this end, we formulate the problem as a SAT model. The generated SAT instances being enormous, we propose some model improvements, both in terms of the number of variables, the number of clauses, and clauses size. These improvements significantly reduce the instances, but at the cost of longer generation time. We thus try to balance instance size vs. generation and solving time. We also achieved some experimental comparisons and we analyzed our various model improvements.

【18】 Thinkback: Task-SpecificOut-of-Distribution Detection 标题：Thinkback：特定于任务的分发中断检测

作者：Lixuan Yang,Dario Rossi 备注：None 链接：https://arxiv.org/abs/2107.06668 摘要：深度学习（Deep Learning，DL）的日益成功最近引发了DL模型在许多不同行业领域的大规模部署。然而，监督模型的一个关键弱点是处理分布外样本的固有困难，即样本属于训练时没有呈现给模型的类。本文提出了一种针对DL模型的分布外检测方法。我们的方法不需要对训练数据进行微调，但是比现有的分布外检测方法更精确。 摘要：The increased success of Deep Learning (DL) has recently sparked large-scale deployment of DL models in many diverse industry segments. Yet, a crucial weakness of supervised model is the inherent difficulty in handling out-of-distribution samples, i.e., samples belonging to classes that were not presented to the model at training time. We propose in this paper a novel way to formulate the out-of-distribution detection problem, tailored for DL models. Our method does not require fine tuning process on training data, yet is significantly more accurate than the state of the art for out-of-distribution detection.

【19】 Plan-Based Relaxed Reward Shaping for Goal-Directed Tasks 标题：基于计划的目标导向任务的轻松奖励形成

作者：Ingmar Schubert,Ozgur S. Oguz,Marc Toussaint 机构： Learning and Intelligent Systems Group, TU Berlin, Germany, Max Planck Institute for Intelligent Systems, Stuttgart, Germany, Machine Learning and Robotics Lab, University of Stuttgart, Germany 备注：None 链接：https://arxiv.org/abs/2107.06661 摘要：在高维状态空间中，强化学习的有效性受到探索问题的限制。这个问题已经解决了使用基于潜力的奖励成形（PB-RS）以前。在目前的工作中，我们介绍了最终体积保持奖励成形（FV-RS）。FV-RS放松了PB-RS严格的最优性保证，保证了长期行为的保持。由于限制较少，FV-RS允许更适合于提高RL算法的采样效率的奖励成形函数。特别地，我们考虑代理可以访问近似计划的设置。在这里，我们使用模拟机器人操作任务的例子来证明基于计划的FV-RS确实可以比基于计划的PB-RS显著提高RL的样本效率。 摘要：In high-dimensional state spaces, the usefulness of Reinforcement Learning (RL) is limited by the problem of exploration. This issue has been addressed using potential-based reward shaping (PB-RS) previously. In the present work, we introduce Final-Volume-Preserving Reward Shaping (FV-RS). FV-RS relaxes the strict optimality guarantees of PB-RS to a guarantee of preserved long-term behavior. Being less restrictive, FV-RS allows for reward shaping functions that are even better suited for improving the sample efficiency of RL algorithms. In particular, we consider settings in which the agent has access to an approximate plan. Here, we use examples of simulated robotic manipulation tasks to demonstrate that plan-based FV-RS can indeed significantly improve the sample efficiency of RL over plan-based PB-RS.

【20】 Trustworthy AI: A Computational Perspective 标题：值得信赖的人工智能：一种计算视角

作者：Haochen Liu,Yiqi Wang,Wenqi Fan,Xiaorui Liu,Yaxin Li,Shaili Jain,Anil K. Jain,Jiliang Tang 机构： Michigan State University, The Hong Kong Polytechnic University 备注：54 pages. arXiv admin note: text overlap with arXiv:1512.04150, arXiv:1602.04938 by other authors 链接：https://arxiv.org/abs/2107.06641 摘要：在过去的几十年里，人工智能技术经历了飞速的发展，改变了每个人的日常生活，深刻地改变了人类社会的进程。开发人工智能的目的是为了造福人类，通过减少人类劳动，为人类生活带来日常便利，促进社会福利。然而，最近的研究和人工智能应用表明，人工智能会对人类造成无意的伤害，比如在安全关键场景中做出不可靠的决定，或者由于无意中歧视某一群体而破坏公平性。因此，值得信赖的人工智能近年来引起了人们的极大关注，这就要求人们认真考虑，避免人工智能可能给人类带来的不利影响，使人类能够充分信任人工智能技术，并与之和谐相处。近年来，人们对可信人工智能进行了大量的研究。在这项调查中，我们提出了一个全面的调查可信的人工智能从计算的角度，以帮助读者了解最新的技术实现可信的人工智能。值得信赖的人工智能是一个庞大而复杂的领域，涉及多个维度。在这项工作中，我们关注实现可信赖人工智能的六个最关键的维度：（i）安全性和稳健性，（ii）非歧视性和公平性，（iii）可解释性，（iv）隐私性，（v）责任性和可审计性，以及（vi）环境福利。对于每个维度，我们根据分类法回顾了最近的相关技术，并总结了它们在实际系统中的应用。我们还讨论了不同维度之间一致和冲突的交互作用，并讨论了值得信赖的人工智能未来研究的潜在方面。 摘要：In the past few decades, artificial intelligence (AI) technology has experienced swift developments, changing everyone's daily life and profoundly altering the course of human society. The intention of developing AI is to benefit humans, by reducing human labor, bringing everyday convenience to human lives, and promoting social good. However, recent research and AI applications show that AI can cause unintentional harm to humans, such as making unreliable decisions in safety-critical scenarios or undermining fairness by inadvertently discriminating against one group. Thus, trustworthy AI has attracted immense attention recently, which requires careful consideration to avoid the adverse effects that AI may bring to humans, so that humans can fully trust and live in harmony with AI technologies. Recent years have witnessed a tremendous amount of research on trustworthy AI. In this survey, we present a comprehensive survey of trustworthy AI from a computational perspective, to help readers understand the latest technologies for achieving trustworthy AI. Trustworthy AI is a large and complex area, involving various dimensions. In this work, we focus on six of the most crucial dimensions in achieving trustworthy AI: (i) Safety & Robustness, (ii) Non-discrimination & Fairness, (iii) Explainability, (iv) Privacy, (v) Accountability & Auditability, and (vi) Environmental Well-Being. For each dimension, we review the recent related technologies according to a taxonomy and summarize their applications in real-world systems. We also discuss the accordant and conflicting interactions among different dimensions and discuss potential aspects for trustworthy AI to investigate in the future.

【21】 You Only Write Thrice: Creating Documents, Computational Notebooks and Presentations From a Single Source 标题：您只需编写三次：从单一来源创建文档、计算笔记本和演示文稿

作者：Kacper Sokol,Peter Flach 机构：Department of Computer Science, University of Bristol, Bristol, United Kingdom 备注：Published at Rethinking ML Papers -- ICLR 2021 Workshop. OpenReview: this https URL Exhibit: this https URL 链接：https://arxiv.org/abs/2107.06639 摘要：学术贸易需要处理以不同格式出版的同一内容的多种变体：手稿、演示文稿、海报和计算笔记本。需要跟踪版本以适应写-审核-反驳-修改的生命周期，这又增加了一层复杂性。我们建议通过在版本控制的环境（如git）中维护单个源文档，添加生成学术界流行的输出格式集合的功能，来显著减少这种负担。为此，我们利用了Jupyter科学计算生态系统中的各种开源工具，并对选定的软件工程概念进行操作。我们提供了一个概念验证工作流，它由Jupyter Book（一个在线文档）、Jupyter Notebook（一个计算叙述）和reveal.js幻灯片组成，这些幻灯片来自单个markdown源文件。托管在GitHub上，我们的方法支持更改跟踪和版本控制，以及基于底层代码问题管理基础设施的透明审查过程。我们的工作流程展示可以在https://so-cool.github.io/you-only-write-thrice/. 摘要：Academic trade requires juggling multiple variants of the same content published in different formats: manuscripts, presentations, posters and computational notebooks. The need to track versions to accommodate for the write--review--rebut--revise life-cycle adds another layer of complexity. We propose to significantly reduce this burden by maintaining a single source document in a version-controlled environment (such as git), adding functionality to generate a collection of output formats popular in academia. To this end, we utilise various open-source tools from the Jupyter scientific computing ecosystem and operationalise selected software engineering concepts. We offer a proof-of-concept workflow that composes Jupyter Book (an online document), Jupyter Notebook (a computational narrative) and reveal.js slides from a single markdown source file. Hosted on GitHub, our approach supports change tracking and versioning, as well as a transparent review process based on the underlying code issue management infrastructure. An exhibit of our workflow can be previewed at https://so-cool.github.io/you-only-write-thrice/.

【22】 Procedural Content Generation using Behavior Trees (PCGBT) 标题：使用行为树生成过程性内容(PCGBT)

作者：Anurag Sarkar,Seth Cooper 机构：Northeastern University 链接：https://arxiv.org/abs/2107.06638 摘要：行为树（Behavior trees，BTs）是一种流行的模拟npc和敌方AI行为的方法，在大量商业游戏中得到了广泛的应用。在本文中，我们不使用BTs来建模游戏代理，而是演示它们在游戏设计代理建模中的应用，将行为定义为执行内容生成任务而不是游戏中的动作。与传统的BTs如何以模块化和动态的方式对行为进行建模类似，BTs for PCG使用于生成部分级别的简单子树能够模块化地组合，以形成用于生成整个级别的更复杂的树以及可以动态改变生成的内容的生成器。我们通过使用BTs来为超级马里奥兄弟、巨型人和Metroid级别以及地下城布局的生成器建模来演示这种方法，并讨论了这种PCGBT范式在未来的应用和扩展的几种方法。 摘要：Behavior trees (BTs) are a popular method of modeling the behavior of NPCs and enemy AI and have found widespread use in a large number of commercial games. In this paper, rather than use BTs to model game-playing agents, we demonstrate their use for modeling game design agents, defining behaviors as executing content generation tasks rather than in-game actions. Similar to how traditional BTs enable modeling behaviors in a modular and dynamic manner, BTs for PCG enable simple subtrees for generating parts of levels to be combined modularly to form more complex trees for generating whole levels as well as generators that can dynamically vary the generated content. We demonstrate this approach by using BTs to model generators for Super Mario Bros., Mega Man and Metroid levels as well as dungeon layouts and discuss several ways in which this PCGBT paradigm could be applied and extended in the future.

【23】 Model-free Reinforcement Learning for Robust Locomotion Using Trajectory Optimization for Exploration 标题：基于探索轨迹优化的鲁棒运动无模型强化学习

作者：Miroslav Bogdanovic,Majid Khadiv,Ludovic Righetti 机构：NewYorkUniversity, eduThis work was supported by New York University 链接：https://arxiv.org/abs/2107.06629 摘要：在这项工作中，我们提出了一个一般的，两阶段的强化学习方法，从一个单一的示范轨迹，到一个强大的政策，可以部署在硬件上没有任何额外的训练。第一阶段以示范为出发点，便于初步探索。在第二阶段，直接优化相关的任务报酬，计算出对环境不确定性具有鲁棒性的策略。我们在一个真实的四足机器人上详细地演示和检验了我们的方法在高动态跳跃和跳跃任务上的性能和鲁棒性。 摘要：In this work we present a general, two-stage reinforcement learning approach for going from a single demonstration trajectory to a robust policy that can be deployed on hardware without any additional training. The demonstration is used in the first stage as a starting point to facilitate initial exploration. In the second stage, the relevant task reward is optimized directly and a policy robust to environment uncertainties is computed. We demonstrate and examine in detail performance and robustness of our approach on highly dynamic hopping and bounding tasks on a real quadruped robot.

【24】 Continuous vs. Discrete Optimization of Deep Neural Networks 标题：深度神经网络的连续优化与离散优化

作者：Omer Elkabetz,Nadav Cohen 机构：Tel Aviv University 链接：https://arxiv.org/abs/2107.06608 摘要：现有的深度学习中的优化分析要么是连续的，集中在梯度流的变体上，要么是离散的，直接处理梯度下降的变体。梯度流是易于理论分析，但是程式化和忽视计算效率。它在多大程度上代表了梯度下降，这在深度学习理论中是一个悬而未决的问题。本文对这一问题进行了研究。将梯度下降作为梯度流初值问题的一种近似数值解法，我们发现其近似程度取决于梯度流轨迹上的曲率。然后，我们证明了在均匀激活的深层神经网络中，梯度流轨迹具有良好的曲率，这表明它们可以很好地用梯度下降来逼近。这一发现使我们能够将对深层线性神经网络上梯度流的分析转化为在随机初始化条件下梯度下降有效收敛到全局最小值的保证。实验表明，在简单的深度神经网络上，采用常规步长的梯度下降确实接近连续极限。我们假设梯度流理论将是揭开深度学习背后奥秘的核心。 摘要：Existing analyses of optimization in deep learning are either continuous, focusing on (variants of) gradient flow, or discrete, directly treating (variants of) gradient descent. Gradient flow is amenable to theoretical analysis, but is stylized and disregards computational efficiency. The extent to which it represents gradient descent is an open question in deep learning theory. The current paper studies this question. Viewing gradient descent as an approximate numerical solution to the initial value problem of gradient flow, we find that the degree of approximation depends on the curvature along the latter's trajectory. We then show that over deep neural networks with homogeneous activations, gradient flow trajectories enjoy favorable curvature, suggesting they are well approximated by gradient descent. This finding allows us to translate an analysis of gradient flow over deep linear neural networks into a guarantee that gradient descent efficiently converges to global minimum almost surely under random initialization. Experiments suggest that over simple deep neural networks, gradient descent with conventional step size is indeed close to the continuous limit. We hypothesize that the theory of gradient flows will be central to unraveling mysteries behind deep learning.

【25】 A Distance Measure for Privacy-preserving Process Mining based on Feature Learning 标题：一种基于特征学习的隐私保护过程挖掘距离度量

作者：Fabian Rösel,Stephan A. Fahrenkrog-Petersen,Han van der Aa,Matthias Weidlich 机构： Humboldt-Universit¨at zu Berlin , Berlin, Germany, University of Mannheim, Mannheim, Germany 备注：Accepted for 17th International Workshop on Business Process Intelligence 链接：https://arxiv.org/abs/2107.06578 摘要：为了能够基于事件日志进行流程分析而不损害流程执行中涉及的个人的隐私，可以对日志进行匿名化。这种匿名化努力转换日志，使其满足可证明的隐私保证，同时在很大程度上保持其用于流程分析的实用性。现有的技术使用简单的语法度量来执行匿名化，以识别合适的转换操作。这样，跟踪中的事件引用的活动的语义就被忽略了，可能导致不相关活动的事件被合并的转换。为了避免这种情况并在匿名化过程中合并活动的语义，我们建议合并一个基于特征学习的距离度量。具体来说，我们展示了事件的嵌入如何为跟踪定义一个距离度量来指导事件日志的匿名化。我们对真实数据的实验表明，与语法化方法相比，使用这种方法的匿名化方法生成的日志在各个维度上都更接近原始日志，因此在流程分析中具有更高的实用性。 摘要：To enable process analysis based on an event log without compromising the privacy of individuals involved in process execution, a log may be anonymized. Such anonymization strives to transform a log so that it satisfies provable privacy guarantees, while largely maintaining its utility for process analysis. Existing techniques perform anonymization using simple, syntactic measures to identify suitable transformation operations. This way, the semantics of the activities referenced by the events in a trace are neglected, potentially leading to transformations in which events of unrelated activities are merged. To avoid this and incorporate the semantics of activities during anonymization, we propose to instead incorporate a distance measure based on feature learning. Specifically, we show how embeddings of events enable the definition of a distance measure for traces to guide event log anonymization. Our experiments with real-world data indicate that anonymization using this measure, compared to a syntactic one, yields logs that are closer to the original log in various dimensions and, hence, have higher utility for process analysis.

【26】 A Note on Learning Rare Events in Molecular Dynamics using LSTM and Transformer 标题：关于用LSTM和Transformer学习分子动力学稀有事件的一个注记

作者：Wenqi Zeng,Siqin Cao,Xuhui Huang,Yuan Yao 机构：)Department of Mathematics, Hong Kong University of Science and Technology, )Department of Chemistry, Hong Kong University of Science and Technology, )Department of Chemical and Biological Engineering, Hong Kong University of Science and Technology 链接：https://arxiv.org/abs/2107.06573 摘要：长期短期记忆（LSTM）等语言模型的递归神经网络已被用来作为建模和预测复杂随机分子系统长期动力学的工具。利用低维反应坐标系的模拟数据，给出了用LSTM学习慢动力学的成功实例。然而，在本报告中我们发现以下三个关键因素显著影响语言模型学习的表现，即反应坐标的维度、时间分辨率和状态划分。将递归神经网络应用于高维分子动力学模拟时，我们发现慢动力学对应的稀有事件可能会被系统的其他快动力学所掩盖，并且无法有效地学习。在这样的条件下，我们发现将构象空间粗粒化为亚稳态，并在估计态间跃迁概率时消除重交事件，可以大大提高分子动力学慢动力学学习的准确性。此外，我们还探讨了其他模型，如Transformer，它们在克服这些问题方面并没有表现出比LSTM优越的性能。因此，利用LSTM和Transformer来学习稀有慢分子动力学事件，在高分辨率数据中选择适当的时间分辨率（即节省MD模拟轨迹的间隔）和状态划分是关键，由于深度神经网络模型可能不会自动分离慢动态和快动态，当两者都存在于相互影响的数据中时。 摘要：Recurrent neural networks for language models like long short-term memory (LSTM) have been utilized as a tool for modeling and predicting long term dynamics of complex stochastic molecular systems. Recently successful examples on learning slow dynamics by LSTM are given with simulation data of low dimensional reaction coordinate. However, in this report we show that the following three key factors significantly affect the performance of language model learning, namely dimensionality of reaction coordinates, temporal resolution and state partition. When applying recurrent neural networks to molecular dynamics simulation trajectories of high dimensionality, we find that rare events corresponding to the slow dynamics might be obscured by other faster dynamics of the system, and cannot be efficiently learned. Under such conditions, we find that coarse graining the conformational space into metastable states and removing recrossing events when estimating transition probabilities between states could greatly help improve the accuracy of slow dynamics learning in molecular dynamics. Moreover, we also explore other models like Transformer, which do not show superior performance than LSTM in overcoming these issues. Therefore, to learn rare events of slow molecular dynamics by LSTM and Transformer, it is critical to choose proper temporal resolution (i.e., saving intervals of MD simulation trajectories) and state partition in high resolution data, since deep neural network models might not automatically disentangle slow dynamics from fast dynamics when both are present in data influencing each other.

【27】 QoS-Aware Scheduling in New Radio Using Deep Reinforcement Learning 标题：基于深度强化学习的新型无线电QoS感知调度

作者：Jakob Stigenberg,Vidit Saxena,Soma Tayamon,Euhanna Ghadimi 机构：Ericsson AB, Stockholm, Sweden, ©,XX IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including 链接：https://arxiv.org/abs/2107.06570 摘要：第五代（5G）新无线（NR）蜂窝网络支持范围广泛的新业务，其中许多业务需要特定于应用的服务质量（QoS），例如在保证的最小比特率或最大可容忍延迟方面。因此，与前几代相比，调度多个并行数据流（每个流都服务于一个独特的应用程序实例）势必成为一项更具挑战性的任务。利用深度强化学习的最新进展，本文提出了一种基于QoS的NR网络深度强化学习代理（QADRA）调度器。与最新的调度启发式算法相比，QADRA调度器在最大化网络性能的同时，显式地优化了QoS满足率。此外，我们针对这些目标对算法进行端到端的训练。我们在一个全尺寸、接近产品级、系统级NR模拟器中评估QADRA，并展示了网络性能的显著提升。在我们的特定评估场景中，QADRA调度器将网络吞吐量提高了30%，同时与最先进的基线相比，保持了由网络服务的VoIP用户的QoS满意率。 摘要：Fifth-generation (5G) New Radio (NR) cellular networks support a wide range of new services, many of which require an application-specific quality of service (QoS), e.g. in terms of a guaranteed minimum bit-rate or a maximum tolerable delay. Therefore, scheduling multiple parallel data flows, each serving a unique application instance, is bound to become an even more challenging task compared to the previous generations. Leveraging recent advances in deep reinforcement learning, in this paper, we propose a QoS-Aware Deep Reinforcement learning Agent (QADRA) scheduler for NR networks. In contrast to state-of-the-art scheduling heuristics, the QADRA scheduler explicitly optimizes for the QoS satisfaction rate while simultaneously maximizing the network performance. Moreover, we train our algorithm end-to-end on these objectives. We evaluate QADRA in a full scale, near-product, system level NR simulator and demonstrate a significant boost in network performance. In our particular evaluation scenario, the QADRA scheduler improves network throughput by 30% while simultaneously maintaining the QoS satisfaction rate of VoIP users served by the network, compared to state-of-the-art baselines.

【28】 MESS: Manifold Embedding Motivated Super Sampling 标题：Mess：流形嵌入激励超抽样

作者：Erik Thordsen,Erich Schubert 链接：https://arxiv.org/abs/2107.06566 摘要：机器学习和数据分析领域的许多方法都依赖于观测数据位于低维流形上的假设。这一假设已经在许多实际数据集上得到了验证。为了利用这种流形假设，通常需要对流形进行局部采样，使其达到一定的密度，以便观察流形的特征。然而，为了增加数据集的内在维数，所需的数据密度引入了对非常大的数据集的需要，从而导致维数灾难的众多面之一。为了克服对局部数据密度要求的增加，我们提出了一个生成虚拟数据点的框架，该虚拟数据点忠实于数据中可观察流形的近似嵌入函数。 摘要：Many approaches in the field of machine learning and data analysis rely on the assumption that the observed data lies on lower-dimensional manifolds. This assumption has been verified empirically for many real data sets. To make use of this manifold assumption one generally requires the manifold to be locally sampled to a certain density such that features of the manifold can be observed. However, for increasing intrinsic dimensionality of a data set the required data density introduces the need for very large data sets, resulting in one of the many faces of the curse of dimensionality. To combat the increased requirement for local data density we propose a framework to generate virtual data points that faithful to an approximate embedding function underlying the manifold observable in the data.

【29】 Domain Generalization with Pseudo-Domain Label for Face Anti-Spoofing 标题：基于伪域标签的人脸防伪领域泛化

作者：Young Eun Kim,Seong-Whan Lee 机构：Department of Artificial Intelligence, Korea University, Seoul, Republic of Korea 链接：https://arxiv.org/abs/2107.06552 摘要：人脸反欺骗（FAS）技术在保护人脸识别系统免受人脸表示攻击方面发挥着重要作用。近年来，FAS领域的许多研究都采用领域泛化技术来解决这一问题。领域泛化的目的是提高泛化性能，更好地检测各种类型的攻击和不可见的攻击。然而，在这一领域的先前研究已经将每个域简单地定义为一个反欺骗的数据集，并将重点放在开发学习技术上。本文提出了一种利用网络中间层的聚类卷积特征统计信息来判断网络域的方法，该方法不需要将网络域标记为数据集。我们不仅使用网络提取特征，而且使用深度估计器来获得伪域标签，而深度估计器在FAS中只是作为辅助任务。在我们的实验中，我们使用了三个数据集进行训练，并使用剩余的一个数据集进行了性能评估，通过总共四组实验来证明所提方法的有效性。 摘要：Face anti-spoofing (FAS) plays an important role in protecting face recognition systems from face representation attacks. Many recent studies in FAS have approached this problem with domain generalization technique. Domain generalization aims to increase generalization performance to better detect various types of attacks and unseen attacks. However, previous studies in this area have defined each domain simply as an anti-spoofing datasets and focused on developing learning techniques. In this paper, we proposed a method that enables network to judge its domain by itself with the clustered convolutional feature statistics from intermediate layers of the network, without labeling domains as datasets. We obtained pseudo-domain labels by not only using the network extracting features, but also using depth estimators, which were previously used only as an auxiliary task in FAS. In our experiments, we trained with three datasets and evaluated the performance with the remaining one dataset to demonstrate the effectiveness of the proposed method by conducting a total of four sets of experiments.

【30】 The I-ADOPT Interoperability Framework for FAIRer data descriptions of biodiversity 标题：I-采用互操作框架实现更公平的生物多样性数据描述

作者：Barbara Magagna,Ilaria Rosati,Maria Stoica,Sirko Schindler,Gwenaelle Moncoiffe,Anusuriya Devaraju,Johannes Peterseil,Robert Huber 机构：Environment Agency Austria, Vienna, Austria, Institute of Research on Terrestrial Ecosystems, National Research Council, Montelibretti, Rome, Italy, University of Colorado, Boulder, Colorado, Institute of Data Science, German Aerospace Center, Jena, Germany 备注：submitted to S4BioDiv 2021: 3rd International Workshop on Semantics for Biodiversity, September 15, 2021, Bozen, Italy 链接：https://arxiv.org/abs/2107.06547 摘要：生物多样性是物种和生态系统内部和之间的差异，对人类福祉和地球的平衡至关重要。它对人类社会的可持续发展至关重要，是一项重要的全球性挑战。生物多样性研究变得越来越数据密集，它处理全球和区域倡议提供的异构和分布式数据，如GBIF、ILTER、LifeWatch、BODC、PANGAEA和TERN，这些倡议采用不同的数据管理实践。特别是，这些举措产生了各种元数据和语义资源，用于描述生物多样性观测，并在数据管理系统中引入了互操作性问题。为了应对这些挑战，一组国际术语提供商和数据中心经理于2019年制定了可观测属性术语互操作描述工作组（I-ADOPT WG），旨在建立一种通用方法来描述观测、测量、计算或导出的内容。基于对现有变量语义表示的广泛分析，工作组最近发布了I-ADOPT框架本体，以促进现有语义资源之间的互操作性，并支持提供机器可读的变量描述，其组件映射到公平词汇表术语。I-ADOPT框架本体定义了一组高级语义组件，可用于描述科学观察中常见的各种模式。这一贡献将侧重于如何应用I-ADOPT框架来表示生物多样性领域中常用的变量。 摘要：Biodiversity, the variation within and between species and ecosystems, is essential for human well-being and the equilibrium of the planet. It is critical for the sustainable development of human society and is an important global challenge. Biodiversity research has become increasingly data-intensive and it deals with heterogeneous and distributed data made available by global and regional initiatives, such as GBIF, ILTER, LifeWatch, BODC, PANGAEA, and TERN, that apply different data management practices. In particular, a variety of metadata and semantic resources have been produced by these initiatives to describe biodiversity observations, introducing interoperability issues across data management systems. To address these challenges, the InteroperAble Descriptions of Observable Property Terminology WG (I-ADOPT WG) was formed by a group of international terminology providers and data center managers in 2019 with the aim to build a common approach to describe what is observed, measured, calculated, or derived. Based on an extensive analysis of existing semantic representations of variables, the WG has recently published the I-ADOPT framework ontology to facilitate interoperability between existing semantic resources and support the provision of machine-readable variable descriptions whose components are mapped to FAIR vocabulary terms. The I-ADOPT framework ontology defines a set of high level semantic components that can be used to describe a variety of patterns commonly found in scientific observations. This contribution will focus on how the I-ADOPT framework can be applied to represent variables commonly used in the biodiversity domain.

【31】 TEACHING -- Trustworthy autonomous cyber-physical applications through human-centred intelligence 标题：以人为本的智能教学--值得信赖的自主网络物理应用

作者：Davide Bacciu,Siranush Akarmazyan,Eric Armengaud,Manlio Bacco,George Bravos,Calogero Calandra,Emanuele Carlini,Antonio Carta,Pietro Cassara,Massimo Coppola,Charalampos Davalas,Patrizio Dazzi,Maria Carmela Degennaro,Daniele Di Sarli,Jürgen Dobaj,Claudio Gallicchio,Sylvain Girbal,Alberto Gotta,Riccardo Groppo,Vincenzo Lomonaco,Georg Macher,Daniele Mazzei,Gabriele Mencagli,Dimitrios Michail,Alessio Micheli,Roberta Peroglio,Salvatore Petroni,Rosaria Potenza,Farank Pourdanesh,Christos Sardianos,Konstantinos Tserpes,Fulvio Tagliabò,Jakob Valtl,Iraklis Varlamis,Omar Veledar 机构：©, IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including 链接：https://arxiv.org/abs/2107.06543 摘要：本文讨论了H2020教学项目对下一代自主应用程序的展望，这些应用程序运行在一个分布的、高度异构的环境中，包括跨越边缘云连续体的虚拟和物理资源。教学提出了以人为中心的观点，利用用户的生理、情感和认知状态作为自主应用程序适应和优化的驱动力。它通过构建一个分布式的、嵌入式的和联合的学习系统，并辅以各种方法和工具来加强其可靠性、安全性和隐私保护。本文讨论了教学方法的主要概念，并指出了与之相关的主要人工智能研究挑战。此外，我们提供了一个教学系统的设计选择，以解决上述挑战的讨论 摘要：This paper discusses the perspective of the H2020 TEACHING project on the next generation of autonomous applications running in a distributed and highly heterogeneous environment comprising both virtual and physical resources spanning the edge-cloud continuum. TEACHING puts forward a human-centred vision leveraging the physiological, emotional, and cognitive state of the users as a driver for the adaptation and optimization of the autonomous applications. It does so by building a distributed, embedded and federated learning system complemented by methods and tools to enforce its dependability, security and privacy preservation. The paper discusses the main concepts of the TEACHING approach and singles out the main AI-related research challenges associated with it. Further, we provide a discussion of the design choices for the TEACHING system to tackle the aforementioned challenges

【32】 Accelerating Distributed K-FAC with Smart Parallelism of Computing and Communication Tasks 标题：利用计算和通信任务的智能并行性加速分布式K-FAC

作者：Shaohuai Shi,Lin Zhang,Bo Li 机构：Department of Computer Science and Engineering, The Hong Kong University of Science and Technology 备注：11 pages. Accepted to IEEE ICDCS 2021 链接：https://arxiv.org/abs/2107.06533 摘要：基于GPU机群的分布式同步随机梯度下降（SGD）训练已被广泛用于加速深部模型的训练过程。然而，SGD在模型参数更新中只使用一阶梯度，这可能需要几天或几周的时间。最近的研究成功地利用了近似二阶信息来加速训练过程，其中Kronecker因子近似曲率（KFAC）算法是训练深度模型最有效的近似算法之一。然而，当利用GPU集群来训练带有分布式KFAC（D-KFAC）的模型时，它需要大量的计算，并且在每次迭代期间引入额外的通信。在这项工作中，我们提出了D-KFAC（SPD-KFAC）与智能并行计算和通信任务，以减少迭代时间。具体来说，1）我们首先描述了D-KFAC的性能瓶颈，2）我们设计并实现了一个用于Kronecker因子计算和动态张量融合通信的流水线机制，3）我们开发了一个负载平衡布局，用于在GPU集群上反转多个矩阵。我们在具有100Gb/s InfiniBand互连的64-GPU集群上进行了实际实验。实验结果表明，我们提出的SPD-KFAC训练方案比现有的算法提高了10%-35%。 摘要：Distributed training with synchronous stochastic gradient descent (SGD) on GPU clusters has been widely used to accelerate the training process of deep models. However, SGD only utilizes the first-order gradient in model parameter updates, which may take days or weeks. Recent studies have successfully exploited approximate second-order information to speed up the training process, in which the Kronecker-Factored Approximate Curvature (KFAC) emerges as one of the most efficient approximation algorithms for training deep models. Yet, when leveraging GPU clusters to train models with distributed KFAC (D-KFAC), it incurs extensive computation as well as introduces extra communications during each iteration. In this work, we propose D-KFAC (SPD-KFAC) with smart parallelism of computing and communication tasks to reduce the iteration time. Specifically, 1) we first characterize the performance bottlenecks of D-KFAC, 2) we design and implement a pipelining mechanism for Kronecker factors computation and communication with dynamic tensor fusion, and 3) we develop a load balancing placement for inverting multiple matrices on GPU clusters. We conduct real-world experiments on a 64-GPU cluster with 100Gb/s InfiniBand interconnect. Experimental results show that our proposed SPD-KFAC training scheme can achieve 10%-35% improvement over state-of-the-art algorithms.

【33】 Learning Algebraic Recombination for Compositional Generalization 标题：用于成分泛化的学习代数重组

作者：Chenyao Liu,Shengnan An,Zeqi Lin,Qian Liu,Bei Chen,Jian-Guang Lou,Lijie Wen,Nanning Zheng,Dongmei Zhang 机构： School of Software, Tsinghua University, Xi’an Jiaotong University, Microsoft Research Asia, Beihang University 备注：ACL Findings 2021 链接：https://arxiv.org/abs/2107.06516 摘要：神经序列模型在语义分析任务中表现出有限的合成泛化能力。组合泛化需要代数重组，即以递归方式动态重组结构化表达式。然而，以往的研究大多集中在词汇单位的重组上，这是代数重组的一个重要而不充分的部分。在本文中，我们提出了一个端到端的神经模型LeAR来学习用于合成泛化的代数重组。关键是将语义分析任务建模为潜在语法代数和语义代数之间的同态，从而鼓励代数重组。具体来说，我们将共同学习两个模块：用于生成潜在语法的编写器和用于分配语义操作的解释器。在两个真实的综合合成综合基准上的实验验证了该模型的有效性。源代码在https://github.com/microsoft/ContextualSP. 摘要：Neural sequence models exhibit limited compositional generalization ability in semantic parsing tasks. Compositional generalization requires algebraic recombination, i.e., dynamically recombining structured expressions in a recursive manner. However, most previous studies mainly concentrate on recombining lexical units, which is an important but not sufficient part of algebraic recombination. In this paper, we propose LeAR, an end-to-end neural model to learn algebraic recombination for compositional generalization. The key insight is to model the semantic parsing task as a homomorphism between a latent syntactic algebra and a semantic algebra, thus encouraging algebraic recombination. Specifically, we learn two modules jointly: a Composer for producing latent syntax, and an Interpreter for assigning semantic operations. Experiments on two realistic and comprehensive compositional generalization benchmarks demonstrate the effectiveness of our model. The source code is publicly available at https://github.com/microsoft/ContextualSP.

【34】 Few-shot Neural Human Performance Rendering from Sparse RGBD Videos 标题：基于稀疏RGBD视频的Few-Shot神经人体性能绘制

作者：Anqi Pang,Xin Chen,Haimin Luo,Minye Wu,Jingyi Yu,Lan Xu 机构： Shanghai Engineering Research Center of Intelligent Vision and Imaging, School of Information, Science and Technology, ShanghaiTech University, Shanghai Institute of Microsystem and Information Technology, Chinese Academy of Sciences 备注：6 pages, 7 figures 链接：https://arxiv.org/abs/2107.06505 摘要：目前针对人类活动的神经绘制方法虽然取得了显著的视图合成效果，但仍然依赖于密集的输入视图或对所有捕获帧的密集训练，导致部署困难和训练过载。然而，如果输入在空间和时间上都是稀疏的，现有的进展将是不适定的。为了填补这一空白，本文提出了一种基于稀疏RGBD输入的Few-Shot神经人体绘制方法（FNHR），该方法利用时间和空间的冗余来生成真实感的人体活动的自由视图输出。我们的FNHR只在关键帧上训练，这些关键帧扩展了输入序列中的运动流形。我们引入了两个分支的神经融合，将神经点渲染和经典的图形纹理管道相结合，在稀疏的关键帧上集成了可靠的观测。此外，我们采用了一种基于补丁的对抗性训练过程，利用局部冗余，避免了对关键帧的过度拟合，得到了很好的细节渲染效果。大量的实验证明了我们的方法在稀疏环境下生成高质量的自由视点结果的有效性。 摘要：Recent neural rendering approaches for human activities achieve remarkable view synthesis results, but still rely on dense input views or dense training with all the capture frames, leading to deployment difficulty and inefficient training overload. However, existing advances will be ill-posed if the input is both spatially and temporally sparse. To fill this gap, in this paper we propose a few-shot neural human rendering approach (FNHR) from only sparse RGBD inputs, which exploits the temporal and spatial redundancy to generate photo-realistic free-view output of human activities. Our FNHR is trained only on the key-frames which expand the motion manifold in the input sequences. We introduce a two-branch neural blending to combine the neural point render and classical graphics texturing pipeline, which integrates reliable observations over sparse key-frames. Furthermore, we adopt a patch-based adversarial training process to make use of the local redundancy and avoids over-fitting to the key-frames, which generates fine-detailed rendering results. Extensive experiments demonstrate the effectiveness of our approach to generate high-quality free view-point results for challenging human performances under the sparse setting.

【35】 Serialized Multi-Layer Multi-Head Attention for Neural Speaker Embedding 标题：神经说话人嵌入的串行化多层多头注意

作者：Hongning Zhu,Kong Aik Lee,Haizhou Li 机构：School of Computing, National University of Singapore, Singapore, Department of Electrical and Computer Engineering, National University of Singapore, Singapore, Institute for Infocomm Research, A⋆STAR, Singapore 备注：Accepted by Interspeech 2021 链接：https://arxiv.org/abs/2107.06493 摘要：该文提出了一种用于文本无关说话人验证中神经说话人嵌入的多层多头注意序列。在以前的工作中，来自一层的帧级特征被聚合以形成一个话语级表示。受Transformer网络的启发，我们提出的方法利用堆叠式自我注意机制的层次结构来获得与说话人更相关的精细特征。序列化注意机制包含一堆自我注意模块，用于创建说话人的固定维度表示。提出的串行化多层多人头注意方法，不是并行地利用多人头注意，而是以串行的方式从一层到下一层聚合和传播注意统计信息。此外，我们使用统计池对每个语句使用输入感知查询。随着层数的增加，神经网络可以学习更多的说话人嵌入。在VoxCeleb1数据集和SITW数据集上的实验结果表明，本文提出的方法在EER和DCF0.01上分别比其他基线方法（包括x-vectors和其他x-vectors 传统注意池方法）提高了9.7%和8.1%。 摘要：This paper proposes a serialized multi-layer multi-head attention for neural speaker embedding in text-independent speaker verification. In prior works, frame-level features from one layer are aggregated to form an utterance-level representation. Inspired by the Transformer network, our proposed method utilizes the hierarchical architecture of stacked self-attention mechanisms to derive refined features that are more correlated with speakers. Serialized attention mechanism contains a stack of self-attention modules to create fixed-dimensional representations of speakers. Instead of utilizing multi-head attention in parallel, the proposed serialized multi-layer multi-head attention is designed to aggregate and propagate attentive statistics from one layer to the next in a serialized manner. In addition, we employ an input-aware query for each utterance with the statistics pooling. With more layers stacked, the neural network can learn more discriminative speaker embeddings. Experiment results on VoxCeleb1 dataset and SITW dataset show that our proposed method outperforms other baseline methods, including x-vectors and other x-vectors conventional attentive pooling approaches by 9.7% in EER and 8.1% in DCF0.01.

【36】 A Convolutional Neural Network Approach to the Classification of Engineering Models 标题：工程模型分类的卷积神经网络方法

作者：Bharadwaj Manda,Pranjal Bhaskare,Ramanathan Muthuganapathy 机构：Indian Institute of Technology Madras 备注：None 链接：https://arxiv.org/abs/2107.06481 摘要：提出了一种基于卷积神经网络（CNNs）的工程模型分类的深度学习方法。近年来，基于gpu的大规模标注数据集和足够的计算能力，人们提出了许多基于深度学习的目标分类方法，特别是在图像和图形模型领域。然而，很少有人提出解决方案的任务功能分类的CAD模型。因此，在本研究中，CAD模型是从工程形状基准（ESB）、国家设计知识库（NDR）收集的，并用使用建模软件创建的较新模型进行扩充，形成一个数据集-“CADNET”。受当前流行的ResNet的启发，提出了在CADNET中使用剩余网络结构。选择加权光场描述子（LFD）作为特征提取方法，生成的图像作为CNN的输入。使用类权重方法解决了数据集中的类不平衡问题。在计算机辅助设计网络（CADNET）上，利用深网络和其它网络结构，对测地距离等其它特征进行了实验。基于LFD的CNN方法使用所提出的网络结构，结合梯度增强，在CADNET上获得了最好的分类精度。 摘要：This paper presents a deep learning approach for the classification of Engineering (CAD) models using Convolutional Neural Networks (CNNs). Owing to the availability of large annotated datasets and also enough computational power in the form of GPUs, many deep learning-based solutions for object classification have been proposed of late, especially in the domain of images and graphical models. Nevertheless, very few solutions have been proposed for the task of functional classification of CAD models. Hence, for this research, CAD models have been collected from Engineering Shape Benchmark (ESB), National Design Repository (NDR) and augmented with newer models created using a modelling software to form a dataset - 'CADNET'. It is proposed to use a residual network architecture for CADNET, inspired by the popular ResNet. A weighted Light Field Descriptor (LFD) scheme is chosen as the method of feature extraction, and the generated images are fed as inputs to the CNN. The problem of class imbalance in the dataset is addressed using a class weights approach. Experiments have been conducted with other signatures such as geodesic distance etc. using deep networks as well as other network architectures on the CADNET. The LFD-based CNN approach using the proposed network architecture, along with gradient boosting yielded the best classification accuracy on CADNET.

【37】 Generative and reproducible benchmarks for comprehensive evaluation of machine learning classifiers 标题：机器学习分类器综合评价的生成性和重现性基准

作者：Patryk Orzechowski,Jason H. Moore 机构：Institute for Biomedical Informatics, University of Pennsylvania, Philadelphia, PA , USA, Department of Automatics and Robotics, AGH University of Science and Technology, al. Mickiewicza ,-, Krakow, Poland, ∗To whom correspondence should be addressed: 备注：12 pages, 3 figures with subfigures 链接：https://arxiv.org/abs/2107.06475 摘要：了解机器学习算法的优缺点对于确定其应用范围至关重要。在这里，我们介绍了多样性和生成性ML基准（DIGEN）-一个综合数据集的集合，用于全面、可复制和可解释的机器学习算法的基准测试，用于二元结果的分类。DIGEN资源由40个数学函数组成，这些函数将连续特征映射到离散端点，以创建合成数据集。这40个函数是使用启发式算法发现的，该算法旨在最大限度地提高多种流行机器学习算法的性能差异，从而为评估和比较新方法提供了一个有用的测试套件。对生成函数的访问有助于理解为什么与其他算法相比，一种方法的性能较差，从而为改进提供了思路。具有大量文档和分析的资源是开源的，可以在GitHub上获得。 摘要：Understanding the strengths and weaknesses of machine learning (ML) algorithms is crucial for determine their scope of application. Here, we introduce the DIverse and GENerative ML Benchmark (DIGEN) - a collection of synthetic datasets for comprehensive, reproducible, and interpretable benchmarking of machine learning algorithms for classification of binary outcomes. The DIGEN resource consists of 40 mathematical functions which map continuous features to discrete endpoints for creating synthetic datasets. These 40 functions were discovered using a heuristic algorithm designed to maximize the diversity of performance among multiple popular machine learning algorithms thus providing a useful test suite for evaluating and comparing new methods. Access to the generative functions facilitates understanding of why a method performs poorly compared to other algorithms thus providing ideas for improvement. The resource with extensive documentation and analyses is open-source and available on GitHub.

【38】 AID-Purifier: A Light Auxiliary Network for Boosting Adversarial Defense 标题：辅助净化器：增强对抗防御的轻型辅助网络

作者：Duhun Hwang,Eunjung Lee,Wonjong Rhee 机构： Our objective is to develop a computationallyEqual contribution 1Department of Intelligence and Informa-tion, Seoul National University, South Korea 2AI Institute 备注：None 链接：https://arxiv.org/abs/2107.06456 摘要：我们提出了一种辅助净化器，它可以通过净化网络的输入来提高对抗训练网络的鲁棒性。辅助净化器是一个辅助网络，作为一个附加到已训练的主分类器。为了使它的计算量小，它被训练成具有二元交叉熵损失的鉴别器。为了从对抗性示例中获得额外有用的信息，架构设计与信息最大化原则密切相关，其中两层主分类网络通过管道连接到辅助网络。利用AVmixup对辅助网络进行训练，以辅助净化的迭代优化过程。AID净化器可以与其他净化器如PixelDefense一起使用，以获得额外的增强效果。总体结果表明，最佳性能的纯化网络可以增强最佳性能的对抗训练网络，其中AID纯化器是一个轻而健壮的竞争候选。 摘要：We propose an AID-purifier that can boost the robustness of adversarially-trained networks by purifying their inputs. AID-purifier is an auxiliary network that works as an add-on to an already trained main classifier. To keep it computationally light, it is trained as a discriminator with a binary cross-entropy loss. To obtain additionally useful information from the adversarial examples, the architecture design is closely related to information maximization principles where two layers of the main classification network are piped to the auxiliary network. To assist the iterative optimization procedure of purification, the auxiliary network is trained with AVmixup. AID-purifier can be used together with other purifiers such as PixelDefend for an extra enhancement. The overall results indicate that the best performing adversarially-trained networks can be enhanced by the best performing purification networks, where AID-purifier is a competitive candidate that is light and robust.

【39】 GREN: Graph-Regularized Embedding Network for Weakly-Supervised Disease Localization in X-ray images 标题：GREN：用于X射线图像弱监督疾病定位的图正则化嵌入网络

作者：Baolian Qi,Gangming Zhao,Xin Wei,Chaowei Fang,Chengwei Pan,Jinpeng Li,Huiguang He,Licheng Jiao 机构： the University ofHong Kong, Jiao are with the School of Artificial Intelligence, XidianUniversity 备注：10 pages, 6 figures 链接：https://arxiv.org/abs/2107.06442 摘要：在胸部X射线图像中定位疾病，很少有仔细的注释，节省了大量的人力。最近的研究采用了创新的弱监督算法，如多实例学习（MIL）和类激活映射（CAM），但是这些方法往往会产生不准确或不完整的区域。其中一个原因是忽略了隐藏在每个图像中的解剖区域之间的关系以及图像之间的关系中的病理含义。在本文中，我们认为跨区域和跨图像关系作为上下文和补偿信息，对于获得更一致和完整的区域至关重要。为了建立这种关系，我们提出了图正则化嵌入网络（Graph Regularized Embedding Network，GREN），它利用图像内和图像间的信息在胸部X线图像上定位疾病。GREN使用预先训练好的U-Net对肺叶进行分割，然后利用图像内图对肺叶之间的图像内关系进行建模，比较不同区域。同时，利用图像间图来建立批内图像之间的关系，对多幅图像进行比较。这个过程模拟了放射科医生的训练和决策过程：比较多个区域和图像进行诊断。为了使神经网络的深嵌入层保留结构信息（在定位任务中很重要），我们使用Hash编码和Hamming距离来计算图，这些图被用作正则化器以便于训练。通过这种方法，我们在NIH胸片数据集上实现了弱监督疾病定位的最新结果。我们的代码可以在线访问。 摘要：Locating diseases in chest X-ray images with few careful annotations saves large human effort. Recent works approached this task with innovative weakly-supervised algorithms such as multi-instance learning (MIL) and class activation maps (CAM), however, these methods often yield inaccurate or incomplete regions. One of the reasons is the neglection of the pathological implications hidden in the relationship across anatomical regions within each image and the relationship across images. In this paper, we argue that the cross-region and cross-image relationship, as contextual and compensating information, is vital to obtain more consistent and integral regions. To model the relationship, we propose the Graph Regularized Embedding Network (GREN), which leverages the intra-image and inter-image information to locate diseases on chest X-ray images. GREN uses a pre-trained U-Net to segment the lung lobes, and then models the intra-image relationship between the lung lobes using an intra-image graph to compare different regions. Meanwhile, the relationship between in-batch images is modeled by an inter-image graph to compare multiple images. This process mimics the training and decision-making process of a radiologist: comparing multiple regions and images for diagnosis. In order for the deep embedding layers of the neural network to retain structural information (important in the localization task), we use the Hash coding and Hamming distance to compute the graphs, which are used as regularizers to facilitate training. By means of this, our approach achieves the state-of-the-art result on NIH chest X-ray dataset for weakly-supervised disease localization. Our codes are accessible online.

【40】 Centralized Model and Exploration Policy for Multi-Agent RL 标题：多Agent RL的集中式模型和探索策略

作者：Qizhen Zhang,Chris Lu,Animesh Garg,Jakob Foerster 机构： University of Toronto, Vector Institute , NVIDIA 链接：https://arxiv.org/abs/2107.06434 摘要：部分可观察、完全合作的多智能体环境下的强化学习（RL）原则上可用于解决许多实际问题，如控制一组救援机器人或一组四直升机。然而，Dec-pomdp比单agent问题更难解决，前者是NEXP完全问题，后者mdp只是P完全问题。因此，目前用于Dec-pomdp的RL算法存在样本复杂度低的问题，从而降低了其对环境交互代价昂贵的实际问题的适用性。我们的关键洞察是，仅使用多项式数量的样本，就可以学习一个集中化的模型，该模型可以推广到不同的策略。然后，我们可以在学习的模型中而不是在真实的系统中优化策略，从而减少环境交互的数量。我们还学习了模型中的集中式探索策略，该策略学习在具有高度模型不确定性的州行动区域中收集额外数据。最后，我们在三个协作通信任务中对所提出的基于模型的算法MARCO进行了实证分析，结果表明该算法的样本效率提高了20倍。 摘要：Reinforcement learning (RL) in partially observable, fully cooperative multi-agent settings (Dec-POMDPs) can in principle be used to address many real-world challenges such as controlling a swarm of rescue robots or a synchronous team of quadcopters. However, Dec-POMDPs are significantly harder to solve than single-agent problems, with the former being NEXP-complete and the latter, MDPs, being just P-complete. Hence, current RL algorithms for Dec-POMDPs suffer from poor sample complexity, thereby reducing their applicability to practical problems where environment interaction is costly. Our key insight is that using just a polynomial number of samples, one can learn a centralized model that generalizes across different policies. We can then optimize the policy within the learned model instead of the true system, reducing the number of environment interactions. We also learn a centralized exploration policy within our model that learns to collect additional data in state-action regions with high model uncertainty. Finally, we empirically evaluate the proposed model-based algorithm, MARCO, in three cooperative communication tasks, where it improves sample efficiency by up to 20x.

【41】 TSCAN : Dialog Structure discovery using SCAN 标题：TSCAN：使用扫描发现对话结构

作者：Apurba Nath,Aayush Kubba 机构：ayush Kubba 链接：https://arxiv.org/abs/2107.06426 摘要：我们可以通过将话语划分成标记的簇来发现对话结构。可以从数据生成这些标签。通常对于对话框，我们需要一个本体，并使用它来发现结构，然而，通过使用无监督分类和自标记，我们能够在没有任何标签或本体的情况下直观地发现这个结构。本文将最近邻语义聚类（SCAN）应用于对话数据。我们使用BERT作为借口任务，并使用了一种自适应的扫描聚类和自标记。这些簇用于确定转移概率并创建对话框结构。用于扫描的自标记方法使得这些结构可以解释为每个簇都有一个标签。由于该方法是无监督的，评估指标是一个挑战，我们使用统计指标作为结构质量的代理 摘要：Can we discover dialog structure by dividing utterances into labelled clusters. Can these labels be generated from the data. Typically for dialogs we need an ontology and use that to discover structure, however by using unsupervised classification and self-labelling we are able to intuit this structure without any labels or ontology. In this paper we apply SCAN (Semantic Clustering using Nearest Neighbors) to dialog data. We used BERT for pretext task and an adaptation of SCAN for clustering and self labeling. These clusters are used to identify transition probabilities and create the dialog structure. The self-labelling method used for SCAN makes these structures interpretable as every cluster has a label. As the approach is unsupervised, evaluation metrics is a challenge, we use statistical measures as proxies for structure quality

【42】 Tourbillon: a Physically Plausible Neural Architecture 标题：陀飞轮：一种物理上可行的神经结构

作者：Mohammadamin Tavakoli,Pierre Baldi,Peter Sadowski 机构：Department of Computer Science, University of California, Irvine, Department of Information and Computer Sciences, University of Hawai‘i at M¯anoa 链接：https://arxiv.org/abs/2107.06424 摘要：在物理神经系统中，反向传播面临许多障碍，包括：需要标记数据、违反局部学习原则、需要对称连接和缺乏模块性。陀飞轮是一个新的架构，解决所有这些限制。它的核心是由一堆循环自动编码器和一个输出层组成。通过循环算法在自监督模式下训练循环自动编码器，通过随机梯度下降在监督模式下训练顶层自动编码器，并选择使用非对称连接在整个堆栈中传播错误信息。虽然Tourbillon架构主要是为了解决物理约束，而不是为了改进深度学习的当前工程应用，但我们在标准基准数据集（包括MNIST、Fashion MNIST和CIFAR10）上展示了它的可行性。我们证明，Tourbillon可以达到与反向传播训练模型相当的性能，并优于其他物理合理的算法，如反馈校准训练模型。 摘要：In a physical neural system, backpropagation is faced with a number of obstacles including: the need for labeled data, the violation of the locality learning principle, the need for symmetric connections, and the lack of modularity. Tourbillon is a new architecture that addresses all these limitations. At its core, it consists of a stack of circular autoencoders followed by an output layer. The circular autoencoders are trained in self-supervised mode by recirculation algorithms and the top layer in supervised mode by stochastic gradient descent, with the option of propagating error information through the entire stack using non-symmetric connections. While the Tourbillon architecture is meant primarily to address physical constraints, and not to improve current engineering applications of deep learning, we demonstrate its viability on standard benchmark datasets including MNIST, Fashion MNIST, and CIFAR10. We show that Tourbillon can achieve comparable performance to models trained with backpropagation and outperform models that are trained with other physically plausible algorithms, such as feedback alignment.

【43】 Monotonicity and Noise-Tolerance in Case-Based Reasoning with Abstract Argumentation (with Appendix) 标题：抽象论证案例推理中的单调性和抗噪性(附附录)

作者：Guilherme Paulino-Passos,Francesca Toni 机构：Imperial College London, Department of Computing 备注：Accepted for KR2021. Includes Appendix. arXiv admin note: substantial text overlap with arXiv:2007.05284 链接：https://arxiv.org/abs/2107.06413 摘要：最近，基于案例推理的抽象论证模型（简称AA{text-}CBR$）被提出，最初是受法律领域的启发，但也适用于不同场景的分类器。然而，$AA{text-}CBR$作为一个推理系统的形式属性仍然没有得到充分的研究。本文着重分析了$AA{text-}CBR$（我们称之为$AA{text-}CBR{suceq}$）的正则版本的非单调性。具体地说，我们证明了$AA{text-}CBR{suceq}$不是谨慎单调的，这是文献中经常考虑的一个性质。然后我们定义一个变量$AA{text-}CBR{suceq}$，它是谨慎单调的。进一步地，我们证明了这种变化等价于使用$AA{text-}CBR{suceq}$和一个受限案例库，该受限案例库由原始案例库中的所有“意外”和“充分”案例组成。作为一个副产品，我们证明了$AA{text-}CBR{succeq}$的这种变化是累积的，合理单调的，并且支持在“非相干”案例库中处理噪声的原则。最后，我们以美国商业秘密领域（一个法律案例库）为例说明$AA{text-}CBR$和谨慎单调性问题。 摘要：Recently, abstract argumentation-based models of case-based reasoning ($AA{text -} CBR$ in short) have been proposed, originally inspired by the legal domain, but also applicable as classifiers in different scenarios. However, the formal properties of $AA{text -} CBR$ as a reasoning system remain largely unexplored. In this paper, we focus on analysing the non-monotonicity properties of a regular version of $AA{text -} CBR$ (that we call $AA{text -} CBR_{succeq}$). Specifically, we prove that $AA{text -} CBR_{succeq}$ is not cautiously monotonic, a property frequently considered desirable in the literature. We then define a variation of $AA{text -} CBR_{succeq}$ which is cautiously monotonic. Further, we prove that such variation is equivalent to using $AA{text -} CBR_{succeq}$ with a restricted casebase consisting of all "surprising" and "sufficient" cases in the original casebase. As a by-product, we prove that this variation of $AA{text -} CBR_{succeq}$ is cumulative, rationally monotonic, and empowers a principled treatment of noise in "incoherent" casebases. Finally, we illustrate $AA{text -} CBR$ and cautious monotonicity questions on a case study on the U.S. Trade Secrets domain, a legal casebase.

【44】 Shortest-Path Constrained Reinforcement Learning for Sparse Reward Tasks 标题：稀疏奖励任务的最短路径约束强化学习

作者：Sungryull Sohn,Sungtae Lee,Jongwook Choi,Harm van Seijen,Mehdi Fatemi,Honglak Lee 机构： 200 3); it has been shown that theEqual contribution 1University of Michigan 2LG AI Research 3Yonsei University 4Microsoft Research 备注：In proceedings of ICML 2021 链接：https://arxiv.org/abs/2107.06405 摘要：我们提出了k-最短路径（k-SP）约束：一种新的对代理轨迹的约束，提高了稀疏奖励mdp中的样本效率。我们证明了任何最优策略都必须满足k-SP约束。值得注意的是，k-SP约束阻止策略沿着非k-SP轨迹探索状态-动作对（例如，来回）。然而，在实际应用中，排除状态-动作对可能会阻碍RL算法的收敛。为了克服这一问题，我们提出了一种新的代价函数来惩罚违反SP约束的策略，而不是完全排除它。在一个表格RL环境下的数值实验表明，SP约束可以显著减小策略的轨迹空间。因此，我们的约束通过抑制重复的探索和利用，实现了更有效的样本学习。我们在MiniGrid、DeepMind Lab、Atari和Fetch上的实验表明，该方法显著改善了近端策略优化（PPO），并优于现有的新颖性探索方法，包括连续控制任务中基于计数的探索，表明它通过防止代理执行冗余操作来提高样本效率。 摘要：We propose the k-Shortest-Path (k-SP) constraint: a novel constraint on the agent's trajectory that improves the sample efficiency in sparse-reward MDPs. We show that any optimal policy necessarily satisfies the k-SP constraint. Notably, the k-SP constraint prevents the policy from exploring state-action pairs along the non-k-SP trajectories (e.g., going back and forth). However, in practice, excluding state-action pairs may hinder the convergence of RL algorithms. To overcome this, we propose a novel cost function that penalizes the policy violating SP constraint, instead of completely excluding it. Our numerical experiment in a tabular RL setting demonstrates that the SP constraint can significantly reduce the trajectory space of policy. As a result, our constraint enables more sample efficient learning by suppressing redundant exploration and exploitation. Our experiments on MiniGrid, DeepMind Lab, Atari, and Fetch show that the proposed method significantly improves proximal policy optimization (PPO) and outperforms existing novelty-seeking exploration methods including count-based exploration even in continuous control tasks, indicating that it improves the sample efficiency by preventing the agent from taking redundant actions.

【45】 Hybrid Memoised Wake-Sleep: Approximate Inference at the Discrete-Continuous Interface 标题：混合记忆觉醒-睡眠：离散-连续界面的近似推理

作者：Tuan Anh Le,Katherine M. Collins,Luke Hewitt,Kevin Ellis,Siddharth N,Samuel J. Gershman,Joshua B. Tenenbaum 机构： 2Cornell University, 3University of Edinburgh and the Alan Turing Institute, 4Harvard UniversityPreprint 链接：https://arxiv.org/abs/2107.06393 摘要：复杂现象的建模通常涉及离散变量和连续变量的使用。这样的设置适用于广泛的问题，从识别时间序列数据的趋势到在图像中执行有效的合成场景理解。在这里，我们提出混合记忆唤醒睡眠（HMWS），一种算法，有效地推理在这种混合离散连续模型。以前的学习方法会受到影响，因为它们需要执行重复的昂贵的内循环离散推理。我们建立在一个最近的方法，记忆唤醒睡眠（MWS）的基础上，它通过记忆离散变量来缓解部分问题，并通过学习用于基于重要性抽样的近似推理和边缘化的单独识别模型来扩展它，以允许一个原则和有效的方法来处理连续变量。我们在GP核学习和3D场景理解领域对HMWS进行了评估，结果表明HMWS的性能优于目前最先进的推理方法。 摘要：Modeling complex phenomena typically involves the use of both discrete and continuous variables. Such a setting applies across a wide range of problems, from identifying trends in time-series data to performing effective compositional scene understanding in images. Here, we propose Hybrid Memoised Wake-Sleep (HMWS), an algorithm for effective inference in such hybrid discrete-continuous models. Prior approaches to learning suffer as they need to perform repeated expensive inner-loop discrete inference. We build on a recent approach, Memoised Wake-Sleep (MWS), which alleviates part of the problem by memoising discrete variables, and extend it to allow for a principled and effective way to handle continuous variables by learning a separate recognition model used for importance-sampling based approximate inference and marginalization. We evaluate HMWS in the GP-kernel learning and 3D scene understanding domains, and show that it outperforms current state-of-the-art inference methods.

【46】 Geometry and Generalization: Eigenvalues as predictors of where a network will fail to generalize 标题：几何学和泛化：特征值作为网络在哪里不能泛化的预测器

作者：Susama Agarwala,Benjamin Dees,Andrew Gearhart,Corey Lowman 链接：https://arxiv.org/abs/2107.06386 摘要：通过训练的权值矩阵的雅可比矩阵，研究了训练的自动编码器对输入空间的变形。在这样做时，我们证明了边界均方误差的点在输入空间，在假设正交性的特征向量。我们还证明了雅可比矩阵的特征值的迹和积是测试点上均方误差的一个很好的预报器。这是一种独立于数据集的方法，用于测试自动编码器对新输入的泛化能力。也就是说，不需要知道网络训练的数据集，只需要训练模型的参数。 摘要：We study the deformation of the input space by a trained autoencoder via the Jacobians of the trained weight matrices. In doing so, we prove bounds for the mean squared errors for points in the input space, under assumptions regarding the orthogonality of the eigenvectors. We also show that the trace and the product of the eigenvalues of the Jacobian matrices is a good predictor of the MSE on test points. This is a dataset independent means of testing an autoencoder's ability to generalize on new input. Namely, no knowledge of the dataset on which the network was trained is needed, only the parameters of the trained model.

【47】 How Much Can CLIP Benefit Vision-and-Language Tasks? 标题：剪辑对视觉和语言任务有多大好处？

作者：Sheng Shen,Liunian Harold Li,Hao Tan,Mohit Bansal,Anna Rohrbach,Kai-Wei Chang,Zhewei Yao,Kurt Keutzer 机构：†University of California, Berkeley, ‡University of California, Los Angeles, ◦University of North Carolina at Chapel Hill 备注：14 pages 链接：https://arxiv.org/abs/2107.06383 摘要：大多数现有的视觉和语言（V&L）模型依赖于预先训练的视觉编码器，使用相对较少的人工标注数据集（与网络爬网数据相比）来感知视觉世界。然而，大规模的预训练通常可以产生更好的泛化性能，例如，CLIP（对比语言图像预训练）在大量的图像字幕对上训练，在各种视觉任务上表现出很强的Zero-Shot能力。为了进一步研究CLIP带来的优势，我们建议在两种典型的场景下，在各种V&L模型中使用CLIP作为视觉编码器：1）将CLIP插入到特定任务的微调中；2） 将CLIP与V&L预训练相结合，并转移到下游任务。我们发现，CLIP明显优于广泛使用的视觉编码器训练领域内的注释数据，如自下而上自顶向下。我们在不同的V&L任务上取得了有竞争力或更好的结果，同时在视觉问答、视觉蕴涵和V&L导航任务上取得了最新的成果。我们在https://github.com/clip-vil/CLIP-ViL. 摘要：Most existing Vision-and-Language (V&L) models rely on pre-trained visual encoders, using a relatively small set of manually-annotated data (as compared to web-crawled data), to perceive the visual world. However, it has been observed that large-scale pretraining usually can result in better generalization performance, e.g., CLIP (Contrastive Language-Image Pre-training), trained on a massive amount of image-caption pairs, has shown a strong zero-shot capability on various vision tasks. To further study the advantage brought by CLIP, we propose to use CLIP as the visual encoder in various V&L models in two typical scenarios: 1) plugging CLIP into task-specific fine-tuning; 2) combining CLIP with V&L pre-training and transferring to downstream tasks. We show that CLIP significantly outperforms widely-used visual encoders trained with in-domain annotated data, such as BottomUp-TopDown. We achieve competitive or better results on diverse V&L tasks, while establishing new state-of-the-art results on Visual Question Answering, Visual Entailment, and V&L Navigation tasks. We release our code at https://github.com/clip-vil/CLIP-ViL.

【48】 Efficient exact computation of the conjunctive and disjunctive decompositions of D-S Theory for information fusion: Translation and extension 标题：信息融合D-S理论合取与析取分解的高效精确计算：翻译与扩展

作者：Maxime Chaveroche,Franck Davoine,Véronique Cherfaoui 机构：Sorbonne University Alliance, Universit´e de technologie de Compiegne, CNRS, Heudiasyc, CS , - , Compiegne Cedex, France 备注：Extension of an article published in the proceedings of the french conference GRETSI 2019 链接：https://arxiv.org/abs/2107.06329 摘要：Dempster-Shafer理论（DST）推广了贝叶斯概率理论，提供了有用的附加信息，但计算量大。为了降低Dempster规则在信息融合中的计算复杂度，人们做了大量的工作。然而，对于作为其他重要信息融合方法核心的证据合取分解和析取分解，却鲜有研究降低计算复杂度。在本文中，我们提出了一种方法，旨在利用这些分解中包含的实际证据（信息）来计算它们。它基于一个新的概念，我们称之为焦点，源自焦点集的概念。有了它，在某些情况下，我们可以将这些计算减少到焦点集数量的线性复杂度。从更广的角度来看，当辨别框架的大小超过几十种可能的状态时，我们的公式有可能是可处理的，这与现有的literature相反。本文扩展（并翻译）了我们在2019年法国GRETSI会议上发表的工作。 摘要：Dempster-Shafer Theory (DST) generalizes Bayesian probability theory, offering useful additional information, but suffers from a high computational burden. A lot of work has been done to reduce the complexity of computations used in information fusion with Dempster's rule. Yet, few research had been conducted to reduce the complexity of computations for the conjunctive and disjunctive decompositions of evidence, which are at the core of other important methods of information fusion. In this paper, we propose a method designed to exploit the actual evidence (information) contained in these decompositions in order to compute them. It is based on a new notion that we call focal point, derived from the notion of focal set. With it, we are able to reduce these computations up to a linear complexity in the number of focal sets in some cases. In a broader perspective, our formulas have the potential to be tractable when the size of the frame of discernment exceeds a few dozen possible states, contrary to the existing litterature. This article extends (and translates) our work published at the french conference GRETSI in 2019.

【49】 On the Performance Analysis of the Adversarial System Variant Approximation Method to Quantify Process Model Generalization 标题：对抗性系统变量逼近方法量化过程模型泛化的性能分析

作者：Julian Theis,Ilia Mokhtarian,Houshang Darabi 机构： Darabi are with University of Illinois atChicago, Department of Mechanical and Industrial Engineering 链接：https://arxiv.org/abs/2107.06319 摘要：流程挖掘算法从事件日志中发现流程模型。由此产生的过程模型应该描述底层系统所有可能的事件序列。泛化是过程模型质量的一个重要维度。泛化度量应该量化过程模型在多大程度上表示事件日志中包含的观察到的事件序列和系统中未观察到的事件序列。文献中的大多数可用度量不能正确地量化过程模型的泛化。最近发表的一种称为敌对系统变量近似的方法[1]利用生成性敌对网络从事件日志中近似系统的底层事件序列分布。虽然这种方法在测量过程模型的泛化方面比现有的方法有更高的性能，但它的实验评估是在理想条件下进行的。实验研究了在有偏和有限事件日志等非理想条件下对抗系统变量逼近的性能。此外，通过实验研究了该方法的采样超参数值对泛化性能的影响。结果表明，需要提高对对抗系统变量近似法工作条件的认识。本文的研究成果也为今后的研究方向提供了参考[1] 泰斯、朱利安和侯尚·达拉比。”量化过程模型泛化之对抗性系统变型近似〉，《IEEE Access 8（2020）:194410-194427。 摘要：Process mining algorithms discover a process model from an event log. The resulting process model is supposed to describe all possible event sequences of the underlying system. Generalization is a process model quality dimension of interest. A generalization metric should quantify the extent to which a process model represents the observed event sequences contained in the event log and the unobserved event sequences of the system. Most of the available metrics in the literature cannot properly quantify the generalization of a process model. A recently published method [1] called Adversarial System Variant Approximation leverages Generative Adversarial Networks to approximate the underlying event sequence distribution of a system from an event log. While this method demonstrated performance gains over existing methods in measuring the generalization of process models, its experimental evaluations have been performed under ideal conditions. This paper experimentally investigates the performance of Adversarial System Variant Approximation under non-ideal conditions such as biased and limited event logs. Moreover, experiments are performed to investigate the originally proposed sampling hyperparameter value of the method on its performance to measure the generalization. The results confirm the need to raise awareness about the working conditions of the Adversarial System Variant Approximation method. The outcomes of this paper also serve to initiate future research directions. [1] Theis, Julian, and Houshang Darabi. "Adversarial System Variant Approximation to Quantify Process Model Generalization." IEEE Access 8 (2020): 194410-194427.

【50】 HDMapNet: An Online HD Map Construction and Evaluation Framework 标题：HDMapNet：一个在线高清地图构建与评估框架

作者：Qi Li,Yue Wang,Yilun Wang,Hang Zhao 机构：Tsinghua University, MIT, Li Auto 链接：https://arxiv.org/abs/2107.06307 摘要：高清地图（HD-map）的构建是自主驾驶的关键问题。该问题通常涉及收集高质量的点云、融合同一场景的多个点云、注释地图元素以及不断更新地图。然而，这个管道需要大量的人力和资源，这限制了它的可扩展性。此外，传统的高清地图与厘米级的精确定位相结合，在许多情况下是不可靠的。在本文中，我们认为在线地图学习是一种基于局部传感器观测动态构建高清地图的方法，与传统的预注释高清地图相比，在线地图学习可以为自动驾驶车辆提供更具可伸缩性的语义和几何先验信息。同时介绍了一种在线地图学习方法HDMapNet。它对来自周围摄像机和/或激光雷达的点云的图像特征进行编码，并预测鸟瞰视图中的矢量化地图元素。我们在nuScenes数据集上对HDMapNet进行了基准测试，结果表明，在所有设置下，它的性能都优于基线方法。值得注意的是，我们基于融合的HDMapNet在所有指标上都比现有的方法高出50%以上。为了加速未来的研究，我们开发了可定制的map学习性能评估指标，包括语义级和实例级的指标。通过介绍这种方法和指标，我们邀请社区来研究这个新的地图学习问题。我们将发布我们的代码和评估工具包，以促进未来的发展。 摘要：High-definition map (HD map) construction is a crucial problem for autonomous driving. This problem typically involves collecting high-quality point clouds, fusing multiple point clouds of the same scene, annotating map elements, and updating maps constantly. This pipeline, however, requires a vast amount of human efforts and resources which limits its scalability. Additionally, traditional HD maps are coupled with centimeter-level accurate localization which is unreliable in many scenarios. In this paper, we argue that online map learning, which dynamically constructs the HD maps based on local sensor observations, is a more scalable way to provide semantic and geometry priors to self-driving vehicles than traditional pre-annotated HD maps. Meanwhile, we introduce an online map learning method, titled HDMapNet. It encodes image features from surrounding cameras and/or point clouds from LiDAR, and predicts vectorized map elements in the bird's-eye view. We benchmark HDMapNet on the nuScenes dataset and show that in all settings, it performs better than baseline methods. Of note, our fusion-based HDMapNet outperforms existing methods by more than 50% in all metrics. To accelerate future research, we develop customized metrics to evaluate map learning performance, including both semantic-level and instance-level ones. By introducing this method and metrics, we invite the community to study this novel map learning problem. We will release our code and evaluation kit to facilitate future development.

【51】 Meta-Optimization of Deep CNN for Image Denoising Using LSTM 标题：基于LSTM的深层CNN图像去噪的Meta优化

作者：Basit O. Alawode,Motaz Alfarraj 机构：King Fahd University of Petroleum and Minerals, Electrical Engineering Department, Dhahran, Saudi Arabia. 链接：https://arxiv.org/abs/2107.06845 摘要：近年来，随着深度学习（DL）在各种任务中的应用，经典技术的性能已经超过了基于DL的技术。因此，DL在去除图像噪声方面也有着同样的应用。特别是，使用深度前馈卷积神经网络（DnCNNs）去噪已被研究。与其他经典的去噪算法相比，它利用了DL技术的进步，如深度结构、残差学习和批量归一化，以获得更好的去噪性能。然而，它的深层架构导致了一组庞大的可训练参数。元优化是一种使算法能够自主学习训练的训练方法。与传统的基于梯度下降的训练方法相比，使用元优化器的训练算法可以使算法获得更好的性能。在这项工作中，我们研究了元优化训练方法在DnCNN去噪算法中的应用，以提高其去噪能力。我们在简单算法上的初步实验揭示了利用元优化训练方法提高DnCNN去噪能力的前景。 摘要：The recent application of deep learning (DL) to various tasks has seen the performance of classical techniques surpassed by their DL-based counterparts. As a result, DL has equally seen application in the removal of noise from images. In particular, the use of deep feed-forward convolutional neural networks (DnCNNs) has been investigated for denoising. It utilizes advances in DL techniques such as deep architecture, residual learning, and batch normalization to achieve better denoising performance when compared with the other classical state-of-the-art denoising algorithms. However, its deep architecture resulted in a huge set of trainable parameters. Meta-optimization is a training approach of enabling algorithms to learn to train themselves by themselves. Training algorithms using meta-optimizers have been shown to enable algorithms to achieve better performance when compared to the classical gradient descent-based training approach. In this work, we investigate the application of the meta-optimization training approach to the DnCNN denoising algorithm to enhance its denoising capability. Our preliminary experiments on simpler algorithms reveal the prospects of utilizing the meta-optimization training approach towards the enhancement of the DnCNN denoising capability.

【52】 Clustering and attention model based for Intelligent Trading 标题：基于聚类和注意力模型的智能交易

作者：Mimansa Rana,Nanxiang Mao,Ming Ao,Xiaohui Wu,Poning Liang,Matloob Khushi 机构：The School of Computer Science, The University of Sydney 链接：https://arxiv.org/abs/2107.06782 摘要：外汇市场在全球金融市场中占有重要地位。外汇交易在给投资者带来高收益机会的同时，也带来一定的风险。自20世纪外汇市场建立以来，汇率预测一直是国内外学者研究的热点问题。由于影响外汇市场因素的复杂性和数量，技术分析无法应对行政干预或突发事件。我们的团队选取了2005年至2021年的几对外币历史数据和衍生技术指标作为数据集，建立了不同的机器学习模型，用于超卖情景下的事件驱动价格预测。 摘要：The foreign exchange market has taken an important role in the global financial market. While foreign exchange trading brings high-yield opportunities to investors, it also brings certain risks. Since the establishment of the foreign exchange market in the 20th century, foreign exchange rate forecasting has become a hot issue studied by scholars from all over the world. Due to the complexity and number of factors affecting the foreign exchange market, technical analysis cannot respond to administrative intervention or unexpected events. Our team chose several pairs of foreign currency historical data and derived technical indicators from 2005 to 2021 as the dataset and established different machine learning models for event-driven price prediction for oversold scenario.

【53】 M5 Competition Uncertainty: Overdispersion, distributional forecasting, GAMLSS and beyond 标题：M5竞争不确定性：过度分散、分布预测、GAMLSS及以后

作者：Florian Ziel 机构：University of Duisburg-Essen 链接：https://arxiv.org/abs/2107.06675 摘要：M5竞争不确定性跟踪旨在对沃尔玛数千种零售商品的销售进行概率预测。我们发现，M5竞争数据面临着强烈的过度分散和零星需求，尤其是零需求。我们讨论由此产生的建模问题，有关充分的概率预测这样的计数数据过程。遗憾的是，大多数流行的预测方法（如lightgbm和xgboost-GBMs）由于考虑了目标函数，无法处理数据特征。分布预测为解决这些问题提供了一种合适的建模方法。GAMLSS框架允许使用低维分布进行灵活的概率预测。通过对负二项分布的位置和尺度参数的建模，说明了GAMLSS方法如何应用于M5竞争数据。最后，我们讨论了分布式建模的软件包及其缺点，如R包gamlss及其扩展包，以及（深层）分布式预测库，如TensorFlow Probability。 摘要：The M5 competition uncertainty track aims for probabilistic forecasting of sales of thousands of Walmart retail goods. We show that the M5 competition data faces strong overdispersion and sporadic demand, especially zero demand. We discuss resulting modeling issues concerning adequate probabilistic forecasting of such count data processes. Unfortunately, the majority of popular prediction methods used in the M5 competition (e.g. lightgbm and xgboost GBMs) fails to address the data characteristics due to the considered objective functions. The distributional forecasting provides a suitable modeling approach for to the overcome those problems. The GAMLSS framework allows flexible probabilistic forecasting using low dimensional distributions. We illustrate, how the GAMLSS approach can be applied for the M5 competition data by modeling the location and scale parameter of various distributions, e.g. the negative binomial distribution. Finally, we discuss software packages for distributional modeling and their drawback, like the R package gamlss with its package extensions, and (deep) distributional forecasting libraries such as TensorFlow Probability.

【54】 A Granular Sieving Algorithm for Deterministic Global Optimization 标题：一种求解确定性全局优化的粒度筛选算法

作者：Tao Qian,Lei Dai,Liming Zhang,Zehua Chen 链接：https://arxiv.org/abs/2107.06581 摘要：提出了一种求解欧氏空间中任意路径连通紧集中Lipschitz连续函数全局优化问题的无梯度确定性方法。该方法可视为在目标函数的域和范围内进行同步分析的颗粒筛分方法。利用适用于一元和多元目标函数的简单数学公式，分别在域空间和值域空间中通过紧致集的两个递减序列来确定全局极小值和所有全局极小值。该算法易于实现，计算量适中。该方法是测试广泛的基准函数在文献中。实验结果表明了该算法的有效性和适用性。 摘要：A gradient-free deterministic method is developed to solve global optimization problems for Lipschitz continuous functions defined in arbitrary path-wise connected compact sets in Euclidean spaces. The method can be regarded as granular sieving with synchronous analysis in both the domain and range of the objective function. With straightforward mathematical formulation applicable to both univariate and multivariate objective functions, the global minimum value and all the global minimizers are located through two decreasing sequences of compact sets in, respectively, the domain and range spaces. The algorithm is easy to implement with moderate computational cost. The method is tested against extensive benchmark functions in the literature. The experimental results show remarkable effectiveness and applicability of the algorithm.