cs.LG 方向，今日共计97篇
Graph相关(图学习|图神经网络|图优化等)(8篇)
【1】 Grounded Graph Decoding Improves Compositional Generalization in Question Answering 标题：接地图解码提高问答中的成分概括性 链接：https://arxiv.org/abs/2111.03642
作者：Yu Gai,Paras Jain,Wendi Zhang,Joseph E. Gonzalez,Dawn Song,Ion Stoica 机构：University of California, Berkeley, equal contribution 备注：To be published in Findings of EMNLP 2021. Code available at this https URL 摘要：问答模型难以推广到训练模式的新组合，例如更长的序列或更复杂的测试结构。当前的端到端模型学习平面输入嵌入，这可能会丢失输入语法上下文。以前的方法通过学习置换不变模型来提高泛化能力，但这些方法不能扩展到更复杂的训练测试分割。我们提出了扎根图解码，这是一种通过将结构化预测与注意机制相结合来提高语言表示的合成泛化的方法。接地使模型能够保留中输入的语法信息，从而显著提高复杂输入的泛化能力。通过预测包含查询子句连词的结构化图，我们学习了一种群不变表示，而无需对目标域进行假设。我们的模型在组合自由基问题（CFQ）数据集上显著优于最先进的基线，这是问答中组合概括的一个具有挑战性的基准。此外，我们有效地解决了MCD1分裂，准确率为98%。 摘要：Question answering models struggle to generalize to novel compositions of training patterns, such to longer sequences or more complex test structures. Current end-to-end models learn a flat input embedding which can lose input syntax context. Prior approaches improve generalization by learning permutation invariant models, but these methods do not scale to more complex train-test splits. We propose Grounded Graph Decoding, a method to improve compositional generalization of language representations by grounding structured predictions with an attention mechanism. Grounding enables the model to retain syntax information from the input in thereby significantly improving generalization over complex inputs. By predicting a structured graph containing conjunctions of query clauses, we learn a group invariant representation without making assumptions on the target domain. Our model significantly outperforms state-of-the-art baselines on the Compositional Freebase Questions (CFQ) dataset, a challenging benchmark for compositional generalization in question answering. Moreover, we effectively solve the MCD1 split with 98% accuracy.
【2】 Learning on Random Balls is Sufficient for Estimating (Some) Graph Parameters 标题：随机球上的学习足以估计(某些)图参数 链接：https://arxiv.org/abs/2111.03317
作者：Takanori Maehara,Hoang NT 机构：Facebook AI, London, United Kingdom, Tokyo Tech & RIKEN AIP, Tokyo, Japan 备注：The manuscript is accepted as a poster presentation at NeurIPS 2021. This ArXiv version includes the Appendix 摘要：对图学习方法的理论分析通常假设对输入图进行完全观察。由于实践中的可伸缩性问题，这样的假设对于处理任何大小的图可能都没有用处。在这项工作中，我们发展了一个在部分观测环境（即子图抽样）下的图分类问题的理论框架。借助于图极限理论，我们提出了一种新的图分类模型，该模型适用于随机抽样的子图，并提出了一种新的拓扑结构来表征该模型的可表示性。我们的理论框架有助于对图的小批量学习进行理论验证，并在不假设输入的情况下，在泛化界和大小泛化性方面得到新的学习理论结果。 摘要：Theoretical analyses for graph learning methods often assume a complete observation of the input graph. Such an assumption might not be useful for handling any-size graphs due to the scalability issues in practice. In this work, we develop a theoretical framework for graph classification problems in the partial observation setting (i.e., subgraph samplings). Equipped with insights from graph limit theory, we propose a new graph classification model that works on a randomly sampled subgraph and a novel topology to characterize the representability of the model. Our theoretical framework contributes a theoretical validation of mini-batch learning on graphs and leads to new learning-theoretic results on generalization bounds as well as size-generalizability without assumptions on the input.
【3】 Graph Denoising with Framelet Regularizer 标题：基于Framelet正则化算法的图形去噪 链接：https://arxiv.org/abs/2111.03264
作者：Bingxin Zhou,Ruikun Li,Xuebin Zheng,Yu Guang Wang,Junbin Gao 机构：Gao are with The University of Sydney BusinessSchool 摘要：由于从真实世界收集的图形数据仅仅是无噪声的，因此图形的实际表示应该对噪声具有鲁棒性。现有的研究通常集中在特征平滑上，但没有触及几何结构。此外，大多数工作采用追求全局平滑的L2范数，这限制了图神经网络的表达能力。本文从特征噪声和结构噪声两个方面对图形数据进行正则化，其中目标函数采用交替方向乘子法（ADMM）进行有效求解。所提出的方案允许在不考虑过度平滑的情况下采用多层，并保证收敛到最优解。实证研究证明，与流行的图卷积相比，即使在图被严重污染的情况下，我们的模型也取得了显著更好的性能。 摘要：As graph data collected from the real world is merely noise-free, a practical representation of graphs should be robust to noise. Existing research usually focuses on feature smoothing but leaves the geometric structure untouched. Furthermore, most work takes L2-norm that pursues a global smoothness, which limits the expressivity of graph neural networks. This paper tailors regularizers for graph data in terms of both feature and structure noises, where the objective function is efficiently solved with the alternating direction method of multipliers (ADMM). The proposed scheme allows to take multiple layers without the concern of over-smoothing, and it guarantees convergence to the optimal solutions. Empirical study proves that our model achieves significantly better performance compared with popular graph convolutions even when the graph is heavily contaminated.
【4】 Collaborative Graph Contrastive Learning: Data Augmentation Composition May Not be Necessary for Graph Representation Learning 标题：协作式图形对比学习：图形表示学习可能不需要数据增强组合 链接：https://arxiv.org/abs/2111.03262
作者：Yuxiang Ren,Jiawei Zhang 机构：IFM Lab, Department of Computer Science, Florida State University, FL, USA, University of California, Davis, USA 摘要：无监督图表示学习对于图数据来说是一个非常重要的话题。对比学习和自监督学习在结构化数据的无监督表示学习中的成功激发了类似的图形尝试。目前使用对比损失的无监督图表示学习和预训练主要基于手工制作的增强图数据之间的对比。然而，由于不可预测的不变性，图形数据的扩充仍然没有得到很好的探索。在本文中，我们提出了一种新的协作图神经网络对比学习框架（CGCL），它使用多个图编码器来观察图。从不同视图观察到的特征作为图形编码器之间对比学习的图形增强，避免任何扰动以保证不变性。CGCL能够处理图形级和节点级表示学习。大量的实验证明了CGCL在无监督的图表示学习中的优势，以及在图表示学习中不需要手工制作数据扩充组合。 摘要：Unsupervised graph representation learning is a non-trivial topic for graph data. The success of contrastive learning and self-supervised learning in the unsupervised representation learning of structured data inspires similar attempts on the graph. The current unsupervised graph representation learning and pre-training using the contrastive loss are mainly based on the contrast between handcrafted augmented graph data. However, the graph data augmentation is still not well-explored due to the unpredictable invariance. In this paper, we propose a novel collaborative graph neural networks contrastive learning framework (CGCL), which uses multiple graph encoders to observe the graph. Features observed from different views act as the graph augmentation for contrastive learning between graph encoders, avoiding any perturbation to guarantee the invariance. CGCL is capable of handling both graph-level and node-level representation learning. Extensive experiments demonstrate the advantages of CGCL in unsupervised graph representation learning and the non-necessity of handcrafted data augmentation composition for graph representation learning.
【5】 Augmentations in Graph Contrastive Learning: Current Methodological Flaws & Towards Better Practices 标题：图形对比学习中的增强：当前的方法论缺陷&走向更好的实践 链接：https://arxiv.org/abs/2111.03220
作者：Puja Trivedi,Ekdeep Singh Lubana,Yujun Yan,Yaoqing Yang,Danai Koutra 机构：University of Michigan, University of California, Berkeley 备注：8 pages, 4 figures 摘要：图分类在生物信息学、社会科学、自动假新闻检测、web文档分类等领域有着广泛的应用。在许多实际场景中，包括web规模的应用程序，在这些应用程序中，标签很少或很难获得，无监督学习是一种自然的范例，但它会牺牲性能。最近，对比学习（CL）使得无监督的计算机视觉模型能够很好地与有监督的计算机视觉模型竞争。分析可视化CL框架的理论和实证工作发现，利用大型数据集和领域感知增强功能是框架成功的关键。有趣的是，graph CL框架通常报告高性能，同时使用数量级较小的数据，并使用领域无关的增强（例如，节点或边缘下降、特征扰动），这可能会破坏图形的基本属性。基于这些差异，我们试图确定：（i）为什么现有的graph CL框架在增强能力较弱和数据有限的情况下仍然表现良好；以及（ii）遵守视觉CL原则是否可以提高图形分类任务的性能。通过广泛的分析，我们发现了graph CL文献中常用的图形数据增强和评估协议中存在的缺陷，并为未来的研究和应用提出了改进的实践和健全性检查。我们表明，在小的基准数据集上，图神经网络的归纳偏差可以显著弥补现有框架的局限性。在对相对较大的图形分类任务进行的案例研究中，我们发现常用的领域不可知扩充性能较差，而遵守visual CL中的原则可以显著提高性能。例如，在基于图的文档分类中，可以用于更好的web搜索，我们显示与任务相关的增强将准确率提高20%。 摘要：Graph classification has applications in bioinformatics, social sciences, automated fake news detection, web document classification, and more. In many practical scenarios, including web-scale applications, where labels are scarce or hard to obtain, unsupervised learning is a natural paradigm but it trades off performance. Recently, contrastive learning (CL) has enabled unsupervised computer vision models to compete well against supervised ones. Theoretical and empirical works analyzing visual CL frameworks find that leveraging large datasets and domain aware augmentations is essential for framework success. Interestingly, graph CL frameworks often report high performance while using orders of magnitude smaller data, and employing domain-agnostic augmentations (e.g., node or edge dropping, feature perturbations) that can corrupt the graphs' underlying properties. Motivated by these discrepancies, we seek to determine: (i) why existing graph CL frameworks perform well despite weak augmentations and limited data; and (ii) whether adhering to visual CL principles can improve performance on graph classification tasks. Through extensive analysis, we identify flawed practices in graph data augmentation and evaluation protocols that are commonly used in the graph CL literature, and propose improved practices and sanity checks for future research and applications. We show that on small benchmark datasets, the inductive bias of graph neural networks can significantly compensate for the limitations of existing frameworks. In case studies with relatively larger graph classification tasks, we find that commonly used domain-agnostic augmentations perform poorly, while adhering to principles in visual CL can significantly improve performance. For example, in graph-based document classification, which can be used for better web search, we show task-relevant augmentations improve accuracy by 20%.
【6】 LW-GCN: A Lightweight FPGA-based Graph Convolutional Network Accelerator 标题：LW-GCN：一种基于FPGA的轻量级图形卷积网络加速器 链接：https://arxiv.org/abs/2111.03184
作者：Zhuofu Tao,Chen Wu,Yuan Liang,Lei He 机构：University of California, Los Angeles, USA 备注：17 pages, 9 figures 摘要：图卷积网络（GCN）被引入到非欧几里德图数据的有效处理中。然而，GCN在计算和内存访问方面存在大量的不规则性，这阻碍了传统神经网络加速器的有效使用。此外，现有的专用GCN加速器需要大量内存，难以在资源有限的边缘设备上实现。在这项工作中，我们提出了LW-GCN，这是一种基于FPGA的轻量级加速器，采用软硬件协同设计的过程来解决GCN推理中计算和内存访问的不规则性。LW-GCN将主要的GCN运算分解为稀疏密集矩阵乘法（SDMM）和密集矩阵乘法（DMM）。我们提出了一种新的压缩格式来平衡PEs之间的工作负载并防止数据危害。此外，我们应用数据量化和工作负载平铺，并将GCN推理的SDMM和DMM映射到资源有限的硬件上的统一体系结构上。在Xilinx Kintex-7 FPGA上使用三种流行的数据集对GCN和GraphSAGE进行评估。与现有的CPU、GPU和最先进的基于FPGA的加速器相比，LW-GCN可将延迟分别减少60倍、12倍和1.7倍，并将功率效率分别提高912倍、511倍和3.87倍。此外，与NVIDIA最新的edge GPU Jetson Xavier NX相比，LW-GCN的加速比和能耗分别为32倍和84倍。 摘要：Graph convolutional networks (GCNs) have been introduced to effectively process non-euclidean graph data. However, GCNs incur large amounts of irregularity in computation and memory access, which prevents efficient use of traditional neural network accelerators. Moreover, existing dedicated GCN accelerators demand high memory volumes and are difficult to implement onto resource limited edge devices. In this work, we propose LW-GCN, a lightweight FPGA-based accelerator with a software-hardware co-designed process to tackle irregularity in computation and memory access in GCN inference. LW-GCN decomposes the main GCN operations into sparse-dense matrix multiplication (SDMM) and dense matrix multiplication (DMM). We propose a novel compression format to balance workload across PEs and prevent data hazards. Moreover, we apply data quantization and workload tiling, and map both SDMM and DMM of GCN inference onto a uniform architecture on resource limited hardware. Evaluation on GCN and GraphSAGE are performed on Xilinx Kintex-7 FPGA with three popular datasets. Compared to existing CPU, GPU, and state-of-the-art FPGA-based accelerator, LW-GCN reduces latency by up to 60x, 12x and 1.7x and increases power efficiency by up to 912x., 511x and 3.87x, respectively. Furthermore, compared with NVIDIA's latest edge GPU Jetson Xavier NX, LW-GCN achieves speedup and energy savings of 32x and 84x, respectively.
【7】 Adversarial Attacks on Knowledge Graph Embeddings via Instance Attribution Methods 标题：基于实例属性方法的知识图嵌入对抗性攻击 链接：https://arxiv.org/abs/2111.03120
作者：Peru Bhardwaj,John Kelleher,Luca Costabello,Declan O'Sullivan 机构：Declan O’Sullivan,∗, ADAPT Centre, Trinity College Dublin, Ireland, ADAPT Centre, TU Dublin, Ireland, Accenture Labs, Ireland 备注：2021 Conference on Empirical Methods in Natural Language Processing (EMNLP 2021) 摘要：尽管知识图嵌入（KGE）被广泛使用，但对于可能破坏其预期行为的安全漏洞知之甚少。我们研究了针对链路预测的KGE模型的数据中毒攻击。这些攻击在训练时制造对抗性的添加或删除，以在测试时导致模型失败。为了选择对抗性删除，我们建议使用来自可解释机器学习的模型不可知实例归因方法，该方法识别对神经模型对测试实例的预测最有影响的训练实例。我们使用这些有影响力的三元组作为对抗性删除。我们进一步提出了一种启发式方法来替换每个有影响的三元组中的两个实体中的一个，以生成对抗性添加。我们的实验表明，所提出的策略优于KGE模型上最先进的数据中毒攻击，并将由于攻击而导致的MRR降级提高了62%。 摘要：Despite the widespread use of Knowledge Graph Embeddings (KGE), little is known about the security vulnerabilities that might disrupt their intended behaviour. We study data poisoning attacks against KGE models for link prediction. These attacks craft adversarial additions or deletions at training time to cause model failure at test time. To select adversarial deletions, we propose to use the model-agnostic instance attribution methods from Interpretable Machine Learning, which identify the training instances that are most influential to a neural model's predictions on test instances. We use these influential triples as adversarial deletions. We further propose a heuristic method to replace one of the two entities in each influential triple to generate adversarial additions. Our experiments show that the proposed strategies outperform the state-of-art data poisoning attacks on KGE models and improve the MRR degradation due to the attacks by up to 62% over the baselines.
【8】 My House, My Rules: Learning Tidying Preferences with Graph Neural Networks 标题：我的房子，我的规则：用图形神经网络学习整理偏好 链接：https://arxiv.org/abs/2111.03112
作者：Ivan Kapelyukh,Edward Johns 机构：The Robot Learning Lab, Imperial College London 备注：Published at CoRL 2021. Webpage and video: this https URL 摘要：安排家庭物品的机器人应该根据用户的喜好来做，这本身就是主观的，很难建模。我们提出了NeatNet：一种新的使用图形神经网络层的变分自动编码器结构，它可以通过观察用户如何安排场景从用户那里提取低维潜在偏好向量。给定任何一组对象，然后可以使用该向量生成一个根据用户的空间偏好定制的排列，并使用单词嵌入来概括新对象。我们开发了一个整理模拟器来收集来自75个用户的重新安排示例，并以经验证明我们的方法在各种重新安排场景中始终产生整洁和个性化的安排。 摘要：Robots that arrange household objects should do so according to the user's preferences, which are inherently subjective and difficult to model. We present NeatNet: a novel Variational Autoencoder architecture using Graph Neural Network layers, which can extract a low-dimensional latent preference vector from a user by observing how they arrange scenes. Given any set of objects, this vector can then be used to generate an arrangement which is tailored to that user's spatial preferences, with word embeddings used for generalisation to new objects. We develop a tidying simulator to gather rearrangement examples from 75 users, and demonstrate empirically that our method consistently produces neat and personalised arrangements across a variety of rearrangement scenarios.
Transformer(1篇)
【1】 Context-Aware Transformer Transducer for Speech Recognition 标题：用于语音识别的上下文感知转换器 链接：https://arxiv.org/abs/2111.03250
作者：Feng-Ju Chang,Jing Liu,Martin Radfar,Athanasios Mouchtaris,Maurizio Omologo,Ariya Rastrow,Siegfried Kunzmann 机构：Amazon Alexa 备注：Accepted to ASRU 2021 摘要：端到端（E2E）自动语音识别（ASR）系统通常难以识别不常见的单词，这些单词很少出现在训练数据中。一种有希望的方法是在推理时利用个性化/上下文信息，以提高对这些稀有词的识别精度。在这项工作中，我们提出了一种新的上下文感知Transformer传感器（CATT）网络，该网络通过利用这种上下文信号改进了最先进的基于Transformer的ASR系统。具体来说，我们提出了一个基于多头注意的上下文偏向网络，该网络与其他ASR子网络联合训练。我们探索不同的技术来编码上下文数据并创建最终的注意上下文向量。我们还利用BLSTM和基于预训练的BERT模型对上下文数据进行编码，并指导网络训练。通过使用内部远场数据集，我们表明，使用基于BERT的上下文编码器的CATT提高了基线Transformer传感器的字错误率，并比现有的深层上下文模型分别高出24.2%和19.4%。 摘要：End-to-end (E2E) automatic speech recognition (ASR) systems often have difficulty recognizing uncommon words, that appear infrequently in the training data. One promising method, to improve the recognition accuracy on such rare words, is to latch onto personalized/contextual information at inference. In this work, we present a novel context-aware transformer transducer (CATT) network that improves the state-of-the-art transformer-based ASR system by taking advantage of such contextual signals. Specifically, we propose a multi-head attention-based context-biasing network, which is jointly trained with the rest of the ASR sub-networks. We explore different techniques to encode contextual data and to create the final attention context vectors. We also leverage both BLSTM and pretrained BERT based models to encode contextual data and guide the network training. Using an in-house far-field dataset, we show that CATT, using a BERT based context encoder, improves the word error rate of the baseline transformer transducer and outperforms an existing deep contextual model by 24.2% and 19.4% respectively.
GAN|对抗|攻击|生成相关(7篇)
【1】 A Unified Game-Theoretic Interpretation of Adversarial Robustness 标题：对抗性稳健性的统一博弈论解释 链接：https://arxiv.org/abs/2111.03536
作者：Jie Ren,Die Zhang,Yisen Wang,Lu Chen,Zhanpeng Zhou,Yiting Chen,Xu Cheng,Xin Wang,Meng Zhou,Jie Shi,Quanshi Zhang 机构：a Shanghai Jiao Tong University, Key Lab. of Machine Perception (MoE), School of EECS, Peking University, c Carnegie Mellon University, d Huawei technologies Inc. 备注：arXiv admin note: substantial text overlap with arXiv:2103.07364 摘要：本文提供了一个统一的观点来解释不同的对抗性攻击和防御方法，即DNN输入变量之间的多阶相互作用的观点。基于多阶交互，我们发现对抗性攻击主要影响高阶交互来愚弄DNN。此外，我们发现对抗训练的DNN的鲁棒性来自于特定类别的低阶相互作用。我们的发现提供了一种潜在的方法来统一对抗性干扰和鲁棒性，这可以从原则上解释现有的防御方法。此外，我们的研究结果也修正了以往对逆向学习特征形状偏差的不准确理解。 摘要：This paper provides a unified view to explain different adversarial attacks and defense methods, emph{i.e.} the view of multi-order interactions between input variables of DNNs. Based on the multi-order interaction, we discover that adversarial attacks mainly affect high-order interactions to fool the DNN. Furthermore, we find that the robustness of adversarially trained DNNs comes from category-specific low-order interactions. Our findings provide a potential method to unify adversarial perturbations and robustness, which can explain the existing defense methods in a principle way. Besides, our findings also make a revision of previous inaccurate understanding of the shape bias of adversarially learned features.
【2】 RADAMS: Resilient and Adaptive Alert and Attention Management Strategy against Informational Denial-of-Service (IDoS) Attacks 标题：RADAMS：抗信息拒绝服务(IDOS)攻击的弹性自适应警报和注意管理策略 链接：https://arxiv.org/abs/2111.03463
作者：Linan Huang,Quanyan Zhu 机构： Zhu are with the Department of Electrical and Com-puter Engineering, New York University 摘要：利用人类注意力脆弱性的攻击对网络安全构成了严重威胁。在这项工作中，我们识别并正式定义了一种新类型的主动注意攻击，称为信息拒绝服务（IDoS）攻击，这种攻击会产生大量虚假攻击，使操作员超负荷工作，并在虚假攻击中隐藏真实攻击。我们结合人为因素（例如，专业水平、压力和效率）和经验结果（例如，耶克斯-多德森定律和沉没成本谬误）对操作员的注意力动态及其决策过程以及实时警报监控和检查进行建模。为了帮助人工操作人员及时准确地击退假动作并升级真实攻击，我们开发了一种弹性和自适应的数据驱动警报和注意力管理策略（RADAMS），该策略根据警报的可观察特征选择性地降低警报的重要程度。RADAMS使用强化学习为各种人类操作员和不断演变的IDoS攻击实现定制和可转移的设计。综合建模和理论分析导致产品注意原则（PPoA）、基本限制以及关键人力和经济因素之间的权衡。实验结果证实，该策略优于默认策略，可将IDoS风险降低20%。此外，该策略能够适应成本、攻击频率和人类注意力能力的巨大变化。我们已经认识到一些有趣的现象，如注意风险等价性、攻击者困境和半真理最优攻击策略。 摘要：Attacks exploiting human attentional vulnerability have posed severe threats to cybersecurity. In this work, we identify and formally define a new type of proactive attentional attacks called Informational Denial-of-Service (IDoS) attacks that generate a large volume of feint attacks to overload human operators and hide real attacks among feints. We incorporate human factors (e.g., levels of expertise, stress, and efficiency) and empirical results (e.g., the Yerkes-Dodson law and the sunk cost fallacy) to model the operators' attention dynamics and their decision-making processes along with the real-time alert monitoring and inspection. To assist human operators in timely and accurately dismissing the feints and escalating the real attacks, we develop a Resilient and Adaptive Data-driven alert and Attention Management Strategy (RADAMS) that de-emphasizes alerts selectively based on the alerts' observable features. RADAMS uses reinforcement learning to achieve a customized and transferable design for various human operators and evolving IDoS attacks. The integrated modeling and theoretical analysis lead to the Product Principle of Attention (PPoA), fundamental limits, and the tradeoff among crucial human and economic factors. Experimental results corroborate that the proposed strategy outperforms the default strategy and can reduce the IDoS risk by as much as 20%. Besides, the strategy is resilient to large variations of costs, attack frequencies, and human attention capacities. We have recognized interesting phenomena such as attentional risk equivalency, attacker's dilemma, and the half-truth optimal attack strategy.
【3】 A Deep Learning Generative Model Approach for Image Synthesis of Plant Leaves 标题：一种用于植物叶片图像合成的深度学习产生式模型方法 链接：https://arxiv.org/abs/2111.03388
作者：Alessandrop Benfenati,Davide Bolzi,Paola Causin,Roberto Oberti 机构： Dept. of Environmental Science and Policy, Universita degli Studi di Milano, Milano, Dept. of Mathematics, Universita degli Studi di Milano, Milano, Italy, Dept. of Agricultural and Environmental Sciences - Production, Landscape 摘要：目标。我们通过先进的深度学习（DL）技术自动生成人工叶片图像。我们的目标是为现代作物管理的人工智能应用处理一个训练样本源。这样的应用程序需要大量的数据，而叶图像并不是真正稀缺的，图像收集和注释仍然是一个非常耗时的过程。数据稀缺性可以通过扩充技术解决，扩充技术包括对属于小数据集的样本进行简单转换，但扩充数据的丰富性是有限的：这促使人们寻找替代方法。方法。采用基于DL生成模型的方法，我们提出了一种叶到叶转换（L2L）过程，该过程分为两步：首先，残差变分自动编码器结构从真实图像的二值化骨架开始生成合成叶骨架（叶轮廓和叶脉）。在第二步中，我们通过Pix2pix框架执行翻译，该框架使用条件生成器对抗网络再现叶片的着色，保留形状和脉络模式。后果L2L程序生成具有真实外观的树叶合成图像。我们以定性和定量的方式处理绩效衡量问题；对于后一种评估，我们采用了DL异常检测策略，该策略量化了合成叶片相对于真实样本的异常程度。结论。生成DL方法有可能成为一种新的范例，为计算机辅助应用提供低成本、有意义的合成样本。目前的L2L方法是朝着这一目标迈出的一步，能够生成与真实叶子在定性和定量上具有相关相似性的合成样品。 摘要：Objectives. We generate via advanced Deep Learning (DL) techniques artificial leaf images in an automatized way. We aim to dispose of a source of training samples for AI applications for modern crop management. Such applications require large amounts of data and, while leaf images are not truly scarce, image collection and annotation remains a very time--consuming process. Data scarcity can be addressed by augmentation techniques consisting in simple transformations of samples belonging to a small dataset, but the richness of the augmented data is limited: this motivates the search for alternative approaches. Methods. Pursuing an approach based on DL generative models, we propose a Leaf-to-Leaf Translation (L2L) procedure structured in two steps: first, a residual variational autoencoder architecture generates synthetic leaf skeletons (leaf profile and veins) starting from companions binarized skeletons of real images. In a second step, we perform translation via a Pix2pix framework, which uses conditional generator adversarial networks to reproduce the colorization of leaf blades, preserving the shape and the venation pattern. Results. The L2L procedure generates synthetic images of leaves with a realistic appearance. We address the performance measurement both in a qualitative and a quantitative way; for this latter evaluation, we employ a DL anomaly detection strategy which quantifies the degree of anomaly of synthetic leaves with respect to real samples. Conclusions. Generative DL approaches have the potential to be a new paradigm to provide low-cost meaningful synthetic samples for computer-aided applications. The present L2L approach represents a step towards this goal, being able to generate synthetic samples with a relevant qualitative and quantitative resemblance to real leaves.
【4】 GraN-GAN: Piecewise Gradient Normalization for Generative Adversarial Networks 标题：GRAN-GAN：生成性对抗网络的分段梯度归一化 链接：https://arxiv.org/abs/2111.03162
作者：Vineeth S. Bhaskara,Tristan Aumentado-Armstrong,Allan Jepson,Alex Levinshtein 机构：Samsung AI Centre Toronto, University of Toronto, Vector Institute for AI 备注：WACV 2022 Main Conference Paper (Submitted: 18 Aug 2021, Accepted: 4 Oct 2021) 摘要：现代生成性对抗网络（GAN）主要在鉴别器（或批评者）中使用分段线性激活函数，包括ReLU和LeakyReLU。这种模型学习分段线性映射，其中每个分段处理输入空间的子集，每个子集的梯度是分段常数。在这类鉴别器（或批评家）函数下，我们提出了梯度归一化（GraN），这是一种新的与输入相关的归一化方法，它保证了输入空间中的分段K-Lipschitz约束。与光谱规范化不同，GraN不限制单个网络层的处理，并且与梯度惩罚不同，GraN几乎在任何地方都严格执行分段Lipschitz约束。根据经验，我们在多个数据集（包括CIFAR-10/100、STL-10、LSUN卧室和CelebA）、GAN损失函数和度量中展示了改进的图像生成性能。此外，我们分析了在几个标准GAN中改变经常不协调的Lipschitz常数K，不仅获得了显著的性能提升，而且还发现了K和训练动态之间的联系，特别是在低梯度损失高原，使用通用Adam优化器。 摘要：Modern generative adversarial networks (GANs) predominantly use piecewise linear activation functions in discriminators (or critics), including ReLU and LeakyReLU. Such models learn piecewise linear mappings, where each piece handles a subset of the input space, and the gradients per subset are piecewise constant. Under such a class of discriminator (or critic) functions, we present Gradient Normalization (GraN), a novel input-dependent normalization method, which guarantees a piecewise K-Lipschitz constraint in the input space. In contrast to spectral normalization, GraN does not constrain processing at the individual network layers, and, unlike gradient penalties, strictly enforces a piecewise Lipschitz constraint almost everywhere. Empirically, we demonstrate improved image generation performance across multiple datasets (incl. CIFAR-10/100, STL-10, LSUN bedrooms, and CelebA), GAN loss functions, and metrics. Further, we analyze altering the often untuned Lipschitz constant K in several standard GANs, not only attaining significant performance gains, but also finding connections between K and training dynamics, particularly in low-gradient loss plateaus, with the common Adam optimizer.
【5】 Generating Diverse Realistic Laughter for Interactive Art 标题：为互动艺术创造多样化的现实主义笑声 链接：https://arxiv.org/abs/2111.03146
作者：M. Mehdi Afsar,Eric Park,Étienne Paquette,Gauthier Gidel,Kory W. Mathewson,Eilif Muller 机构：Mila & University of Calgary, Mila & University of Waterloo, Independent Artist, Mila & University of Montreal, DeepMind 备注：Presented at Machine Learning for Creativity and Design workshop at NeurIPS 2021 摘要：我们提出了一个互动艺术项目，通过欢笑的旋律，以及通过先进的笑合成方法创建和探索的联系，使那些因新冠肺炎及其伴随的孤独而隐形的人重新出现。然而，在高质量的听觉合成中无条件地产生人类情感反应的多样性仍然是一个开放的问题，对于这些方法在艺术环境中的应用具有重要意义。我们开发了LaughGANter，这是一种利用生成性对抗网络（GAN）再现人类笑声多样性的方法。在对不同笑声样本的数据集进行训练时，LaughGANter生成不同的高质量笑声样本，并学习适合情感分析和新颖艺术应用（如潜在混合/插值和情感转移）的潜在空间。 摘要：We propose an interactive art project to make those rendered invisible by the COVID-19 crisis and its concomitant solitude reappear through the welcome melody of laughter, and connections created and explored through advanced laughter synthesis approaches. However, the unconditional generation of the diversity of human emotional responses in high-quality auditory synthesis remains an open problem, with important implications for the application of these approaches in artistic settings. We developed LaughGANter, an approach to reproduce the diversity of human laughter using generative adversarial networks (GANs). When trained on a dataset of diverse laughter samples, LaughGANter generates diverse, high quality laughter samples, and learns a latent space suitable for emotional analysis and novel artistic applications such as latent mixing/interpolation and emotional transfer.
【6】 Generative Adversarial Network for Probabilistic Forecast of Random Dynamical System 标题：随机动力系统概率预测的产生式对抗性网络 链接：https://arxiv.org/abs/2111.03126
作者：Kyongmin Yeo,Zan Li,Wesley M. Gifford 机构： Rensselaer Polytechnic Institute 摘要：我们提出了一个深度学习模型，用于无分布假设的随机动力系统的数据驱动模拟。深度学习模型包括一个用于学习时间推进结构的递归神经网络和一个用于学习和采样随机动力系统概率分布的生成对抗网络。尽管生成性对抗网络为复杂概率分布建模提供了强有力的工具，但如果没有适当的正则化，训练往往会失败。在这里，我们提出了一种基于序列推理问题一致性条件的生成性对抗网络正则化策略。首先，使用最大平均差异（MMD）来加强随机过程的条件分布和边缘分布之间的一致性。然后，使用MMD或多个鉴别器对多步预测的边缘分布进行正则化。利用三个具有复杂噪声结构的随机过程研究了该模型的行为。 摘要：We present a deep learning model for data-driven simulations of random dynamical systems without a distributional assumption. The deep learning model consists of a recurrent neural network, which aims to learn the time marching structure, and a generative adversarial network to learn and sample from the probability distribution of the random dynamical system. Although generative adversarial networks provide a powerful tool to model a complex probability distribution, the training often fails without a proper regularization. Here, we propose a regularization strategy for a generative adversarial network based on consistency conditions for the sequential inference problems. First, the maximum mean discrepancy (MMD) is used to enforce the consistency between conditional and marginal distributions of a stochastic process. Then, the marginal distributions of the multiple-step predictions are regularized by using MMD or from multiple discriminators. The behavior of the proposed model is studied by using three stochastic processes with complex noise structures.
【7】 Physics-Guided Generative Adversarial Networks for Sea Subsurface Temperature Prediction 标题：物理制导的产生式对抗性海温预测网络 链接：https://arxiv.org/abs/2111.03064
作者：Yuxin Meng,Eric Rigall,Xueen Chen,Feng Gao,Junyu Dong,Sheng Chen 机构： OceanUniversity of China, Dong is with Institute of Advanced Oceanography 备注：This work has been accepted by IEEE TNNLS for publication. Our codes and datasets are available at this https URL 摘要：海洋地下温度是水生野生动物、水下动力学和海洋表面传热的重要组成部分，在气候变化中受到全球变暖的影响。现有的研究通常基于基于物理的数值模型或基于数据的模型。物理建模和机器学习传统上被认为是海洋地下温度预测任务的两个不相关领域，具有非常不同的科学范式（物理驱动和数据驱动）。然而，我们认为这两种方法是相辅相成的。物理建模方法可以提供超越观测条件的外推潜力，而数据驱动方法在适应数据方面具有灵活性，并且能够检测到意外模式。这两种方法的结合非常有吸引力，并提供了潜在的性能改进。在本文中，我们提出了一种基于生成对抗网络（GAN）和数值模式相结合的海洋地下温度预测新框架。首先，使用GAN基模型学习数值模型中表面温度和目标次表面温度之间的简化物理关系。然后，利用观测数据对GAN基模型参数进行校正，以获得更好的预测结果。我们通过预测南海的每日海洋地下温度来评估所提出的框架。大量实验表明，与现有的先进方法相比，该框架是有效的。 摘要：Sea subsurface temperature, an essential component of aquatic wildlife, underwater dynamics and heat transfer with the sea surface, is affected by global warming in climate change. Existing research is commonly based on either physics-based numerical models or data based models. Physical modeling and machine learning are traditionally considered as two unrelated fields for the sea subsurface temperature prediction task, with very different scientific paradigms (physics-driven and data-driven). However, we believe both methods are complementary to each other. Physical modeling methods can offer the potential for extrapolation beyond observational conditions, while data-driven methods are flexible in adapting to data and are capable of detecting unexpected patterns. The combination of both approaches is very attractive and offers potential performance improvement. In this paper, we propose a novel framework based on generative adversarial network (GAN) combined with numerical model to predict sea subsurface temperature. First, a GAN-based model is used to learn the simplified physics between the surface temperature and the target subsurface temperature in numerical model. Then, observation data are used to calibrate the GAN-based model parameters to obtain better prediction. We evaluate the proposed framework by predicting daily sea subsurface temperature in the South China sea. Extensive experiments demonstrate the effectiveness of the proposed framework compared to existing state-of-the-art methods.
半/弱/无/有监督|不确定性|主动学习(5篇)
【1】 Mixtures of Laplace Approximations for Improved Post-Hoc Uncertainty in Deep Learning 标题：用于改进深度学习中后自组织不确定性的混合拉普拉斯近似 链接：https://arxiv.org/abs/2111.03577
作者：Runa Eschenhagen,Erik Daxberger,Philipp Hennig,Agustinus Kristiadi 机构：University of Tübingen, University of Cambridge, MPI for Intelligent Systems, Tübingen 备注：Bayesian Deep Learning Workshop, NeurIPS 2021 摘要：深度神经网络容易对异常值做出过度自信的预测。贝叶斯神经网络和深度集成都在一定程度上缓解了这个问题。在这项工作中，我们的目标是结合两种方法的优点，提出用高斯混合模型进行预测，该模型由独立训练的深层神经网络的拉普拉斯近似加权和组成。该方法可用于任何一组预先训练的网络，并且与常规集合相比只需要较小的计算和内存开销。我们从理论上验证了我们的方法缓解了“远离”训练数据的过度自信，并在经验上与标准不确定性量化基准的最新基线进行了比较。 摘要：Deep neural networks are prone to overconfident predictions on outliers. Bayesian neural networks and deep ensembles have both been shown to mitigate this problem to some extent. In this work, we aim to combine the benefits of the two approaches by proposing to predict with a Gaussian mixture model posterior that consists of a weighted sum of Laplace approximations of independently trained deep neural networks. The method can be used post hoc with any set of pre-trained networks and only requires a small computational and memory overhead compared to regular ensembles. We theoretically validate that our approach mitigates overconfidence "far away" from the training data and empirically compare against state-of-the-art baselines on standard uncertainty quantification benchmarks.
【2】 S-multi-SNE: Semi-Supervised Classification and Visualisation of Multi-View Data 标题：S-MULTI-SNE：多视图数据的半监督分类与可视化 链接：https://arxiv.org/abs/2111.03519
作者：Theodoulos Rodosthenous,Vahid Shahrezaei,Marina Evangelou 机构：Department of Mathematics, Imperial College London, London, SW,AZ, UK 备注：13 pages; 3 figures; 3 tables 摘要：多个领域的研究正在发布越来越多的多视图数据。这种类型的数据对应于多个数据视图，每个视图表示同一组样本的不同方面。我们最近提出了multi-SNE，这是t-SNE的一个扩展，它可以生成多视图数据的单一可视化。multi-SNE方法提供了样本的低维嵌入，这些样本是通过不同的数据视图进行迭代更新而生成的。在这里，我们进一步将multi-SNE扩展到一种半监督方法，该方法通过将标记信息视为额外的数据视图来对未标记样本进行分类。通过将这两种方法应用于具有不同挑战的各种多视图数据集，我们深入研究了multi-SNE及其扩展S-multi-SNE的性能、局限性和优势。我们发现，通过包含标签信息，样本的投影得到了极大的改善，并伴随着强大的分类性能。 摘要：An increasing number of multi-view data are being published by studies in several fields. This type of data corresponds to multiple data-views, each representing a different aspect of the same set of samples. We have recently proposed multi-SNE, an extension of t-SNE, that produces a single visualisation of multi-view data. The multi-SNE approach provides low-dimensional embeddings of the samples, produced by being updated iteratively through the different data-views. Here, we further extend multi-SNE to a semi-supervised approach, that classifies unlabelled samples by regarding the labelling information as an extra data-view. We look deeper into the performance, limitations and strengths of multi-SNE and its extension, S-multi-SNE, by applying the two methods on various multi-view datasets with different challenges. We show that by including the labelling information, the projection of the samples improves drastically and it is accompanied by a strong classification performance.
【3】 Automated Supervised Feature Selection for Differentiated Patterns of Care 标题：用于差异化护理模式的自动监督特征选择 链接：https://arxiv.org/abs/2111.03495
作者：Catherine Wanjiru,William Ogallo,Girmaw Abebe Tadesse,Charles Wachira,Isaiah Onando Mulang',Aisha Walcott-Bryant 机构：Carnegie Mellon University Africa, IBM Research Africa, Isaiah Onando Mulang’ 摘要：使用几种最先进的特征选择技术开发了一个自动特征选择管道，以选择用于区分护理模式（DPOC）的最佳特征。管道包括三种类型的特征选择技术；过滤器、包装器和嵌入式方法来选择前K个功能。使用五种不同的二元因变量数据集，并选择其不同的前K个最优特征。选择的特征在现有的多维子集扫描（MDS）中进行测试，其中记录了最不正常的亚群、最不正常的子集、倾向评分和测量效果，以测试其性能。该性能与在MDSS管道中使用数据集中的所有协变量后获得的四个类似指标进行了比较。我们发现，尽管使用了不同的特征选择技术，但在确定要使用的技术时，数据分布是需要注意的关键。 摘要：An automated feature selection pipeline was developed using several state-of-the-art feature selection techniques to select optimal features for Differentiating Patterns of Care (DPOC). The pipeline included three types of feature selection techniques; Filters, Wrappers and Embedded methods to select the top K features. Five different datasets with binary dependent variables were used and their different top K optimal features selected. The selected features were tested in the existing multi-dimensional subset scanning (MDSS) where the most anomalous subpopulations, most anomalous subsets, propensity scores, and effect of measures were recorded to test their performance. This performance was compared with four similar metrics gained after using all covariates in the dataset in the MDSS pipeline. We found out that despite the different feature selection techniques used, the data distribution is key to note when determining the technique to use.
【4】 Supervised Advantage Actor-Critic for Recommender Systems 标题：推荐系统的受监督优势执行者-批评者 链接：https://arxiv.org/abs/2111.03474
作者：Xin Xin,Alexandros Karatzoglou,Ioannis Arapakis,Joemon M. Jose 机构：School of Computer Science, Shandong University, China, Google Research, London, UK, Telefonica Research, Barcelona, Spain, School of Computing Science, University of Glasgow, UK 备注：9 pages, 4 figures, In Proceedings of the 15th ACM International Conference on Web Search and Data Mining (WSDM '22), February 21-25, 2022, Phoenix, Arizona. arXiv admin note: text overlap with arXiv:2006.05779 摘要：通过奖励信号将基于会话或顺序的推荐转换为强化学习（RL），是实现累积利润最大化的推荐系统（RS）的一个有前途的研究方向。然而，由于诸如非策略训练、巨大的行动空间和缺乏足够的奖励信号等挑战，在RS设置中直接使用RL算法是不切实际的。最近的RS的RL方法试图通过结合RL和（自）监督顺序学习来应对这些挑战，但仍有一定的局限性。例如，由于缺少负奖励信号，Q值的估计往往偏向于正值。此外，Q值还严重依赖于序列的特定时间戳。针对上述问题，我们提出了负采样策略来训练RL分量，并将其与有监督序列学习相结合。我们称这种方法为监督负Q学习（SNQN）。基于抽样（消极）行动（项目），我们可以计算积极行动相对于平均情况的“优势”，这可以进一步用作学习监督序列部分的归一化权重。这导致了另一个学习框架：监督优势参与者-批评家（SA2C）。我们用四种最先进的顺序推荐模型实例化了SNQN和SA2C，并在两个真实数据集上进行了实验。实验结果表明，该方法比现有的监督方法和自监督RL方法具有更好的性能。代码将是开源的。 摘要：Casting session-based or sequential recommendation as reinforcement learning (RL) through reward signals is a promising research direction towards recommender systems (RS) that maximize cumulative profits. However, the direct use of RL algorithms in the RS setting is impractical due to challenges like off-policy training, huge action spaces and lack of sufficient reward signals. Recent RL approaches for RS attempt to tackle these challenges by combining RL and (self-)supervised sequential learning, but still suffer from certain limitations. For example, the estimation of Q-values tends to be biased toward positive values due to the lack of negative reward signals. Moreover, the Q-values also depend heavily on the specific timestamp of a sequence. To address the above problems, we propose negative sampling strategy for training the RL component and combine it with supervised sequential learning. We call this method Supervised Negative Q-learning (SNQN). Based on sampled (negative) actions (items), we can calculate the "advantage" of a positive action over the average case, which can be further utilized as a normalized weight for learning the supervised sequential part. This leads to another learning framework: Supervised Advantage Actor-Critic (SA2C). We instantiate SNQN and SA2C with four state-of-the-art sequential recommendation models and conduct experiments on two real-world datasets. Experimental results show that the proposed approaches achieve significantly better performance than state-of-the-art supervised methods and existing self-supervised RL methods . Code will be open-sourced.
【5】 Generalized Radiograph Representation Learning via Cross-supervision between Images and Free-text Radiology Reports 标题：基于图像和自由文本放射学报告交叉监督的广义X线图像表示学习 链接：https://arxiv.org/abs/2111.03452
作者：Hong-Yu Zhou,Xiaoyu Chen,Yinghao Zhang,Ruibang Luo,Liansheng Wang,Yizhou Yu 机构： Department of Computer Science, The University of Hong Kong, Pokfulam, Hong Kong, Department of Computer Science, Xiamen University, Xiamen, China, † These authors contributed equally 备注：Technical Report 摘要：预训练为深学习支持的放射影像分析的成功奠定了基础。它通过在源域上进行大规模的全监督或自监督学习来学习可转移的图像表示。然而，有监督的预训练需要一个复杂且劳动密集的两阶段人工辅助注释过程，而自我监督学习无法与有监督范式竞争。为了解决这些问题，我们提出了一种称为“审查自由文本报告以供监督”（REFERED）的交叉监督方法，该方法从放射照片附带的原始放射报告中获取自由监督信号。所提出的方法采用了视觉转换器，旨在从每个患者研究的多个视图中学习联合表示。在极其有限的监督下，在4个著名的X射线数据集上，Reference优于其转移学习和自我监督学习对手。此外，reference甚至超过了基于具有人工辅助结构标签的射线照片源域的方法。因此，REFERED有可能取代规范的训练前方法。 摘要：Pre-training lays the foundation for recent successes in radiograph analysis supported by deep learning. It learns transferable image representations by conducting large-scale fully-supervised or self-supervised learning on a source domain. However, supervised pre-training requires a complex and labor intensive two-stage human-assisted annotation process while self-supervised learning cannot compete with the supervised paradigm. To tackle these issues, we propose a cross-supervised methodology named REviewing FreE-text Reports for Supervision (REFERS), which acquires free supervision signals from original radiology reports accompanying the radiographs. The proposed approach employs a vision transformer and is designed to learn joint representations from multiple views within every patient study. REFERS outperforms its transfer learning and self-supervised learning counterparts on 4 well-known X-ray datasets under extremely limited supervision. Moreover, REFERS even surpasses methods based on a source domain of radiographs with human-assisted structured labels. Thus REFERS has the potential to replace canonical pre-training methodologies.
迁移|Zero/Few/One-Shot|自适应(4篇)
【1】 Solving Traffic4Cast Competition with U-Net and Temporal Domain Adaptation 标题：基于U网和时域自适应的Traffic4Cast竞争解决方案 链接：https://arxiv.org/abs/2111.03421
作者：Vsevolod Konyakhin,Nina Lukashina,Aleksei Shpilman 机构：ITMO University, Saint Petersburg, Russia, JetBrains Research, HSE University 备注：Conference on Neural Information Processing Systems (NeurIPS 2021) Traffic4cast Competition 摘要：在本技术报告中，我们介绍了Traffic4Cast 2021核心挑战的解决方案，其中要求参与者根据前一小时的信息，在4个不同的城市开发算法，预测60分钟前的交通状态。与之前举办的竞赛不同，今年的挑战集中于新冠病毒-19大流行导致的交通量时域变化。继U-Net过去的成功之后，我们利用它来预测未来的交通地图。此外，我们还探讨了使用预先训练好的编码器，如DenseNet和EfficientNet，并采用多域自适应技术来对抗域转移。我们的解决方案在决赛中排名第三。该守则可于https://github.com/jbr-ai-labs/traffic4cast-2021. 摘要：In this technical report, we present our solution to the Traffic4Cast 2021 Core Challenge, in which participants were asked to develop algorithms for predicting a traffic state 60 minutes ahead, based on the information from the previous hour, in 4 different cities. In contrast to the previously held competitions, this year's challenge focuses on the temporal domain shift in traffic due to the COVID-19 pandemic. Following the past success of U-Net, we utilize it for predicting future traffic maps. Additionally, we explore the usage of pre-trained encoders such as DenseNet and EfficientNet and employ multiple domain adaptation techniques to fight the domain shift. Our solution has ranked third in the final competition. The code is available at https://github.com/jbr-ai-labs/traffic4cast-2021.
【2】 Meta-Forecasting by combining Global DeepRepresentations with Local Adaptation 标题：全局深度表示与局部适应相结合的元预测 链接：https://arxiv.org/abs/2111.03418
作者：Riccardo Grazzi,Valentin Flunkert,David Salinas,Tim Januschowski,Matthias Seeger,Cedric Archambeau 机构：IIT and UCL, Amazon Web Services 摘要：虽然经典的时间序列预测是孤立地考虑单个时间序列，但基于深度学习的最新进展表明，从大量相关时间序列中联合学习可以提高预测精度。然而，当对样本外时间序列进行建模时，这些方法的准确性会受到很大影响，与经典预测方法相比，它们的适用性受到很大限制。为了弥补这一差距，我们采用了时间序列预测问题的元学习观点。我们介绍了一种新的预测方法，称为元全局局部自回归（Meta-GLAR），该方法通过以封闭形式学习从递归神经网络（RNN）产生的表示到一步超前预测的映射来适应每个时间序列。关键的是，RNN的参数是通过封闭形式的自适应机制通过反向传播跨多个时间序列学习的。在我们广泛的实证评估中，我们表明，我们的方法在样本外预测精度方面与早期工作中报告的最新水平具有竞争力。 摘要：While classical time series forecasting considers individual time series in isolation, recent advances based on deep learning showed that jointly learning from a large pool of related time series can boost the forecasting accuracy. However, the accuracy of these methods suffers greatly when modeling out-of-sample time series, significantly limiting their applicability compared to classical forecasting methods. To bridge this gap, we adopt a meta-learning view of the time series forecasting problem. We introduce a novel forecasting method, called Meta Global-Local Auto-Regression (Meta-GLAR), that adapts to each time series by learning in closed-form the mapping from the representations produced by a recurrent neural network (RNN) to one-step-ahead forecasts. Crucially, the parameters ofthe RNN are learned across multiple time series by backpropagating through the closed-form adaptation mechanism. In our extensive empirical evaluation we show that our method is competitive with the state-of-the-art in out-of-sample forecasting accuracy reported in earlier work.
【3】 Adaptive Low-Pass Filtering using Sliding Window Gaussian Processes 标题：基于滑动窗口高斯过程的自适应低通滤波 链接：https://arxiv.org/abs/2111.03617
作者：Alejandro J. Ordóñez-Conejo,Armin Lederer,Sandra Hirche 机构： Ord´o˜nez-Conejo is with Costa Rica Institute of Technology, com 2Armin Lederer and Sandra Hirche are with the Department of Electricaland Computer Engineering, Technical University of Munich 摘要：当通过物理传感器测量信号时，它们会受到噪声的干扰。为了降低噪声，通常使用低通滤波器来衰减输入信号中的高频分量，而不管它们是来自噪声还是来自实际信号。因此，必须仔细调整低通滤波器，以避免信号严重恶化。这种调整需要事先了解信号，这在强化学习或基于学习的控制等应用中通常不可用。为了克服这一限制，我们提出了一种基于高斯过程回归的自适应低通滤波器。通过考虑之前观测的恒定窗口，可以实现足够快的更新和预测，以用于实际过滤应用。此外，超参数的在线优化导致低通行为的自适应，因此无需事先调整。我们证明了该方法的估计误差是一致有界的，并在多个仿真中证明了该方法的灵活性和有效性。 摘要：When signals are measured through physical sensors, they are perturbed by noise. To reduce noise, low-pass filters are commonly employed in order to attenuate high frequency components in the incoming signal, regardless if they come from noise or the actual signal. Therefore, low-pass filters must be carefully tuned in order to avoid significant deterioration of the signal. This tuning requires prior knowledge about the signal, which is often not available in applications such as reinforcement learning or learning-based control. In order to overcome this limitation, we propose an adaptive low-pass filter based on Gaussian process regression. By considering a constant window of previous observations, updates and predictions fast enough for real-world filtering applications can be realized. Moreover, the online optimization of hyperparameters leads to an adaptation of the low-pass behavior, such that no prior tuning is necessary. We show that the estimation error of the proposed method is uniformly bounded, and demonstrate the flexibility and efficiency of the approach in several simulations.
【4】 Improved Regret Analysis for Variance-Adaptive Linear Bandits and Horizon-Free Linear Mixture MDPs 标题：改进的方差自适应线性带和无界线性混合MDP的遗憾分析 链接：https://arxiv.org/abs/2111.03289
作者：Yeoneung Kim,Insoon Yang,Kwang-Sung Jun 机构：Seoul National University, University of Arizona 摘要：在在线学习问题中，利用低方差在获得严格的性能保证方面发挥着重要作用，但由于方差通常是未知的，因此具有挑战性。最近，Zhang et al.（2021）取得了相当大的进展，他们在不知道方差的情况下获得了线性强盗的方差自适应后悔界和线性混合马尔可夫决策过程（MDP）的无地平线后悔界。在本文中，我们提出了新的分析，大大提高了他们的遗憾界限。对于线性土匪，我们实现$tilde O（d^{1.5}sqrt{sum{k}^ksigma{k^2} d^2）$，其中$d$是特征的维度，$k$是时间范围，$sigma_k^2$是时间步长$k$处的噪声方差，$tilde O$忽略多段对数依赖性，这是$d^3$改善的因素。对于线性混合MDP，我们实现了$tilde O（d^{1.5}sqrt{K} d^3）$的无水平遗憾界，其中$d$是基本模型的数量，$K$是剧集的数量。这是一个前导项提高$d^3$的系数，低阶项提高$d^6$。我们的分析严格依赖于一个新的椭圆势“计数”引理。这个引理允许基于剥离的后悔分析，这可能是独立的兴趣。 摘要：In online learning problems, exploiting low variance plays an important role in obtaining tight performance guarantees yet is challenging because variances are often not known a priori. Recently, a considerable progress has been made by Zhang et al. (2021) where they obtain a variance-adaptive regret bound for linear bandits without knowledge of the variances and a horizon-free regret bound for linear mixture Markov decision processes (MDPs). In this paper, we present novel analyses that improve their regret bounds significantly. For linear bandits, we achieve $tilde O(d^{1.5}sqrt{sum_{k}^K sigma_k^2} d^2)$ where $d$ is the dimension of the features, $K$ is the time horizon, and $sigma_k^2$ is the noise variance at time step $k$, and $tilde O$ ignores polylogarithmic dependence, which is a factor of $d^3$ improvement. For linear mixture MDPs, we achieve a horizon-free regret bound of $tilde O(d^{1.5}sqrt{K} d^3)$ where $d$ is the number of base models and $K$ is the number of episodes. This is a factor of $d^3$ improvement in the leading term and $d^6$ in the lower order term. Our analysis critically relies on a novel elliptical potential `count' lemma. This lemma allows a peeling-based regret analysis, which can be of independent interest.
强化学习(3篇)
【1】 Perturbational Complexity by Distribution Mismatch: A Systematic Analysis of Reinforcement Learning in Reproducing Kernel Hilbert Space 标题：分布失配引起的扰动复杂性：再生核Hilbert空间强化学习的系统分析 链接：https://arxiv.org/abs/2111.03469
作者：Jihao Long,Jiequn Han 机构：Program of Applied and Computational Mathematics, Princeton University, Center for Computational Mathematics, Flatiron Institute 摘要：由于在不确定环境下处理高维空间中的函数逼近比较困难，现有的强化学习（RL）理论分析大多局限于表格设置或线性模型。这项工作通过分析一般再生核希尔BERT空间（RKHS）中的RL，为这一挑战提供了一个新的视角。我们考虑一个家庭的马尔可夫决策过程$ MathCAL{M}$，其中奖赏函数位于RKHS的单位球中，并且转移概率位于给定的任意集合中。我们通过分布不匹配$Delta_{mathcal{M}（epsilon）$定义了一个称为微扰复杂性的量，以表征可容许状态-作用分布空间的复杂性，以响应标度为$epsilon$的RKHS中的微扰。我们证明了$Delta{mathcal{M}（epsilon）$给出了所有可能算法的误差下界和RL问题的两个特定算法（拟合奖励和拟合Q迭代）的上界。因此，$DeltauMathcal{M}（epsilon）$相对于$epsilon$的衰减度量了$mathcal{M}$上RL问题的难度。我们进一步提供了一些具体的例子，并讨论了$Delta{mathcal{M}（epsilon）$在这些例子中是否快速衰减。作为副产品，我们证明了当报酬函数位于高维RKHS中时，即使转移概率已知且作用空间有限，RL问题仍然可能遭受维数灾难。 摘要：Most existing theoretical analysis of reinforcement learning (RL) is limited to the tabular setting or linear models due to the difficulty in dealing with function approximation in high dimensional space with an uncertain environment. This work offers a fresh perspective into this challenge by analyzing RL in a general reproducing kernel Hilbert space (RKHS). We consider a family of Markov decision processes $mathcal{M}$ of which the reward functions lie in the unit ball of an RKHS and transition probabilities lie in a given arbitrary set. We define a quantity called perturbational complexity by distribution mismatch $Delta_{mathcal{M}}(epsilon)$ to characterize the complexity of the admissible state-action distribution space in response to a perturbation in the RKHS with scale $epsilon$. We show that $Delta_{mathcal{M}}(epsilon)$ gives both the lower bound of the error of all possible algorithms and the upper bound of two specific algorithms (fitted reward and fitted Q-iteration) for the RL problem. Hence, the decay of $Delta_mathcal{M}(epsilon)$ with respect to $epsilon$ measures the difficulty of the RL problem on $mathcal{M}$. We further provide some concrete examples and discuss whether $Delta_{mathcal{M}}(epsilon)$ decays fast or not in these examples. As a byproduct, we show that when the reward functions lie in a high dimensional RKHS, even if the transition probability is known and the action space is finite, it is still possible for RL problems to suffer from the curse of dimensionality.
【2】 Control of a fly-mimicking flyer in complex flow using deep reinforcement learning 标题：基于深度强化学习的复杂流动仿生飞翔飞片控制 链接：https://arxiv.org/abs/2111.03454
作者：Seungpyo Hong,Sejin Kim,Donghyun You 机构：Department of Mechanical Engineering, Pohang University of Science and Technology, Cheongam-Ro, Nam-Gu, Pohang, Gyeongbuk , South Korea 备注：53 pages, 13 figures, 1 algorithm, 1 table 摘要：建立了一个计算流体结构动力学（CFD-CSD）和深度强化学习（deep RL）的集成框架，用于复杂流动中的飞行尺度柔性翼飞行器的控制。复杂流动中飞片的动力学是高度非定常和非线性的，这使得动力学建模具有挑战性。因此，传统的动力学建模控制方法不足以调节如此复杂的动力学。因此，在本研究中，提出了一个综合框架，其中求解了流体和结构的整个控制方程，以生成飞片的控制策略。为了让深度RL成功地学习控制策略，需要准确和充足的动力学数据。然而，满足复杂动力学数据的质量和数量是极其困难的，因为一般来说，更精确的数据成本更高。在本研究中，我们提出了两种策略来应对这一困境。为了获得准确的数据，采用CFD-CSD精确预测动力学。为了获得足够的数据，设计了一种新的数据复制方法，在保持动态的同时，将获得的数据复制到各种情况下。利用这些数据，该框架学习了各种流动条件下的控制策略，并在复杂流场中对飞片进行了控制。 摘要：An integrated framework of computational fluid-structural dynamics (CFD-CSD) and deep reinforcement learning (deep-RL) is developed for control of a fly-scale flexible-winged flyer in complex flow. Dynamics of the flyer in complex flow is highly unsteady and nonlinear, which makes modeling the dynamics challenging. Thus, conventional control methodologies, where the dynamics is modeled, are insufficient for regulating such complicated dynamics. Therefore, in the present study, the integrated framework, in which the whole governing equations for fluid and structure are solved, is proposed to generate a control policy for the flyer. For the deep-RL to successfully learn the control policy, accurate and ample data of the dynamics are required. However, satisfying both the quality and quantity of the data on the intricate dynamics is extremely difficult since, in general, more accurate data are more costly. In the present study, two strategies are proposed to deal with the dilemma. To obtain accurate data, the CFD-CSD is adopted for precisely predicting the dynamics. To gain ample data, a novel data reproduction method is devised, where the obtained data are replicated for various situations while conserving the dynamics. With those data, the framework learns the control policy in various flow conditions and the learned policy is shown to have remarkable performance in controlling the flyer in complex flow fields.
【3】 Learning to Cooperate with Unseen Agent via Meta-Reinforcement Learning 标题：通过元强化学习学习与看不见的Agent合作 链接：https://arxiv.org/abs/2111.03431
作者：Rujikorn Charakorn,Poramate Manoonpong,Nat Dilokthanakul 备注：Accepted as extended abstract at AAMAS 2021 摘要：特设团队问题描述了一个代理必须与以前看不见的代理合作以实现共同目标的情况。要使代理在这些场景中成功，它必须具有适当的合作技能。通过使用领域知识设计代理的行为，可以将协作技能实现到代理中。然而，在复杂领域中，领域知识可能不可用。因此，如何从数据中直接学习合作技能是值得探讨的。在这项工作中，我们将元强化学习（meta-RL）公式应用于特定团队协作问题。我们的实证结果表明，这种方法可以在两种不同合作环境下产生鲁棒的合作主体：社会遵从性和语言解释。（这是扩展摘要版本的全文。） 摘要：Ad hoc teamwork problem describes situations where an agent has to cooperate with previously unseen agents to achieve a common goal. For an agent to be successful in these scenarios, it has to have a suitable cooperative skill. One could implement cooperative skills into an agent by using domain knowledge to design the agent's behavior. However, in complex domains, domain knowledge might not be available. Therefore, it is worthwhile to explore how to directly learn cooperative skills from data. In this work, we apply meta-reinforcement learning (meta-RL) formulation in the context of the ad hoc teamwork problem. Our empirical results show that such a method could produce robust cooperative agents in two cooperative environments with different cooperative circumstances: social compliance and language interpretation. (This is a full paper of the extended abstract version.)
医学相关(3篇)
【1】 Nonnegative Matrix Factorization to understand Spatio-Temporal Traffic Pattern Variations during COVID-19: A Case Study 标题：非负矩阵分解在理解冠状病毒时空交通模式变化中的应用研究 链接：https://arxiv.org/abs/2111.03592
作者：Anandkumar Balasubramaniam,Thirunavukarasu Balasubramaniam,Rathinaraja Jeyaraj,Anand Paul,Richi Nayak 机构： School of Computer Science & Engineering, Kyungpook National University, Daegu, South Korea., School of Computer Science and Centre for Data Science, Queensland University of, Technology, Brisbane, Australia. 备注：Accepted in the 19th Australasian Data Mining Conference 2021 摘要：由于智能交通系统（ITS）的快速发展和道路上车辆数量的不断增加，产生了丰富的道路交通数据。从这些数据中了解时空交通模式至关重要，并有效地帮助交通规划、道路建设等。然而，了解新冠肺炎流行期间的交通模式非常具有挑战性和重要性，因为人们和车辆的出行行为模式存在巨大差异。在本文中，进行了一个案例研究，以了解在新冠病毒-19期间时空交通模式的变化。我们应用非负矩阵分解（NMF）来引出模式。NMF模型输出基于2019年和2020年期间观察到的时空模式行为进行分析，这两年分别是大流行之前和大流行情况期间，在英国。分析的时空交通模式变化行为的输出将有助于智能交通系统的交通管理领域，以及与道路交通相关的流行病或不可避免情景的各个阶段的管理。 摘要：Due to the rapid developments in Intelligent Transportation System (ITS) and increasing trend in the number of vehicles on road, abundant of road traffic data is generated and available. Understanding spatio-temporal traffic patterns from this data is crucial and has been effectively helping in traffic plannings, road constructions, etc. However, understanding traffic patterns during COVID-19 pandemic is quite challenging and important as there is a huge difference in-terms of people's and vehicle's travel behavioural patterns. In this paper, a case study is conducted to understand the variations in spatio-temporal traffic patterns during COVID-19. We apply nonnegative matrix factorization (NMF) to elicit patterns. The NMF model outputs are analysed based on the spatio-temporal pattern behaviours observed during the year 2019 and 2020, which is before pandemic and during pandemic situations respectively, in Great Britain. The outputs of the analysed spatio-temporal traffic pattern variation behaviours will be useful in the fields of traffic management in Intelligent Transportation System and management in various stages of pandemic or unavoidable scenarios in-relation to road traffic.
【2】 A Retrospective Analysis using Deep-Learning Models for Prediction of Survival Outcome and Benefit of Adjuvant Chemotherapy in Stage II/III Colorectal Cancer 标题：应用深度学习模型预测II/III期结直肠癌患者生存结局和辅助化疗效益的回顾性分析 链接：https://arxiv.org/abs/2111.03532
作者：Xingyu Li,Jitendra Jonnagaddala,Shuhua Yang,Hong Zhang,Xu Steven Xu 机构： Department of Statistics and Finance, School of Management, University of Science and Technology of China, Hefei, Anhui, China;, School of Population Health, UNSW Sydney, NSW, Australia, Data ScienceTranslational Research, Genmab Inc., Princeton 摘要：大多数早期结直肠癌（CRC）患者可以通过单独手术治愈，只有某些高危早期结直肠癌患者受益于辅助化疗。然而，很少有经验证的生物标志物可用于准确预测术后化疗的生存效益。我们开发了一种新的深度学习算法（CRCNet），使用分子和细胞肿瘤学（MCO）的全幻灯片图像预测辅助化疗对II/III期CRC的生存益处。我们通过交叉验证对CRCNet进行了内部验证，并使用癌症基因组图谱（TCGA）中的独立队列对其进行了外部验证。我们发现，CRCNet不仅可以准确预测生存预后，而且可以预测辅助化疗的疗效。CRCNet发现辅助化疗对高危亚组的益处最大，化疗患者的生存期显著延长。相反，在CRCNet低风险和中等风险亚组中观察到最小的化疗益处。因此，CRCNet在指导II/III期大肠癌的治疗方面具有潜在的巨大用途。 摘要：Most early-stage colorectal cancer (CRC) patients can be cured by surgery alone, and only certain high-risk early-stage CRC patients benefit from adjuvant chemotherapies. However, very few validated biomarkers are available to accurately predict survival benefit from postoperative chemotherapy. We developed a novel deep-learning algorithm (CRCNet) using whole-slide images from Molecular and Cellular Oncology (MCO) to predict survival benefit of adjuvant chemotherapy in stage II/III CRC. We validated CRCNet both internally through cross-validation and externally using an independent cohort from The Cancer Genome Atlas (TCGA). We showed that CRCNet can accurately predict not only survival prognosis but also the treatment effect of adjuvant chemotherapy. The CRCNet identified high-risk subgroup benefits from adjuvant chemotherapy most and significant longer survival is observed among chemo-treated patients. Conversely, minimal chemotherapy benefit is observed in the CRCNet low- and medium-risk subgroups. Therefore, CRCNet can potentially be of great use in guiding treatments for Stage II/III CRC.
【3】 Fighting COVID-19 in the Dark: Methodology for Improved Inference Using Homomorphically Encrypted DNN 标题：在黑暗中对抗冠状病毒：使用同态加密DNN改进推理的方法学 链接：https://arxiv.org/abs/2111.03362
作者：Moran Baruch,Lev Greenberg,Guy Moshkowich 机构：IBM Research, Bar-Ilan University 摘要：隐私保护深度神经网络（DNN）推理在医疗、金融和零售等不同监管行业中是必要的。最近，同态加密（HE）已被用作一种方法，以实现分析，同时解决隐私问题。他能够对加密数据进行安全预测。然而，HE的使用存在一些挑战，包括DNN尺寸限制和缺乏对某些操作类型的支持。最值得注意的是，一些HE方案不支持常用的ReLU激活。我们提出了一种结构化的方法，用二次多项式激活替换ReLU。为了解决精度下降问题，我们使用一个预先训练的模型，该模型使用诸如“可训练激活”函数和知识提取等技术训练另一个HE友好模型。我们在AlexNet体系结构上演示了我们的方法，使用胸部X射线和CT数据集检测新冠病毒-19。我们的实验表明，通过使用我们的方法，使用ReLU训练的模型和HE友好模型的F1分数和准确性之间的差距缩小到仅1.1-5.3%的降级范围内。 摘要：Privacy-preserving deep neural network (DNN) inference is a necessity in different regulated industries such as healthcare, finance, and retail. Recently, homomorphic encryption (HE) has been used as a method to enable analytics while addressing privacy concerns. HE enables secure predictions over encrypted data. However, there are several challenges related to the use of HE, including DNN size limitations and the lack of support for some operation types. Most notably, the commonly used ReLU activation is not supported under some HE schemes. We propose a structured methodology to replace ReLU with a quadratic polynomial activation. To address the accuracy degradation issue, we use a pre-trained model that trains another HE-friendly model, using techniques such as "trainable activation" functions and knowledge distillation. We demonstrate our methodology on the AlexNet architecture, using the chest X-Ray and CT datasets for COVID-19 detection. Our experiments show that by using our approach, the gap between the F1 score and accuracy of the models trained with ReLU and the HE-friendly model is narrowed down to within a mere 1.1 - 5.3 percent degradation.
蒸馏|知识提取(3篇)
【1】 AUTOKD: Automatic Knowledge Distillation Into A Student Architecture Family 标题：AUTOKD：学生建筑家庭的自动知识提炼 链接：https://arxiv.org/abs/2111.03555
作者：Roy Henha Eyono,Fabio Maria Carlucci,Pedro M Esperança,Binxin Ru,Phillip Torr 机构：McGill University,MILA,Huawei Noah’s Ark Lab,Facebook Research,University of Oxford 备注：12 pages, 8 figures 摘要：深度学习的最新成果一直在稳步提高，这在很大程度上是由于使用了更大的模型。然而，设备硬件限制限制了广泛使用，导致最先进的模型与可有效部署在小型设备上的模型之间存在巨大的性能差距。虽然知识提炼（KD）理论上可以让小的学生模型模仿大的教师模型，但在实践中选择好的学生体系结构需要大量的人力资源。神经体系结构搜索（NAS）似乎是这个问题的自然解决方案，但大多数方法都可能效率低下，因为大多数计算都花费在比较从相同分布中采样的体系结构上，性能差异可以忽略不计。在本文中，我们建议寻找一个学生体系结构的家庭，这些学生体系结构具有向给定教师学习的能力。我们的方法AutoKD由贝叶斯优化提供支持，探索了一个灵活的基于图形的搜索空间，使我们能够自动学习最佳的学生架构分布和KD参数，同时与现有的最先进技术相比，样本效率提高了20倍。我们在3个数据集上评估了我们的方法；特别是在大图像上，我们在使用3倍内存和10倍参数的情况下达到了教师的表现。最后，虽然AutoKD使用传统的KD损耗，但其性能优于使用手工设计学生的更高级KD变体。 摘要：State-of-the-art results in deep learning have been improving steadily, in good part due to the use of larger models. However, widespread use is constrained by device hardware limitations, resulting in a substantial performance gap between state-of-the-art models and those that can be effectively deployed on small devices. While Knowledge Distillation (KD) theoretically enables small student models to emulate larger teacher models, in practice selecting a good student architecture requires considerable human expertise. Neural Architecture Search (NAS) appears as a natural solution to this problem but most approaches can be inefficient, as most of the computation is spent comparing architectures sampled from the same distribution, with negligible differences in performance. In this paper, we propose to instead search for a family of student architectures sharing the property of being good at learning from a given teacher. Our approach AutoKD, powered by Bayesian Optimization, explores a flexible graph-based search space, enabling us to automatically learn the optimal student architecture distribution and KD parameters, while being 20x more sample efficient compared to existing state-of-the-art. We evaluate our method on 3 datasets; on large images specifically, we reach the teacher performance while using 3x less memory and 10x less parameters. Finally, while AutoKD uses the traditional KD loss, it outperforms more advanced KD variants using hand-designed students.
【2】 Distilling Heterogeneity: From Explanations of Heterogeneous Treatment Effect Models to Interpretable Policies 标题：提炼异质性：从异质治疗效果模型的解释到可解释的政策 链接：https://arxiv.org/abs/2111.03267
作者：Han Wu,Sarah Tan,Weiwei Li,Mia Garrard,Adam Obeng,Drew Dimmery,Shaun Singh,Hanson Wang,Daniel Jiang,Eytan Bakshy 备注：A short version was presented at MIT CODE 2021 摘要：互联网公司越来越多地使用机器学习模型来创建个性化的策略，为每个人分配最佳预测治疗。它们通常来自预测个体水平治疗效果的黑盒异质治疗效果（HTE）模型。在本文中，我们主要关注（1）HTE模型的学习解释；（2） 学习规定治疗任务的可解释政策。我们还提出了指导树，这是一种在不丢失可解释性的情况下集成多个可解释策略的方法。这些基于规则的可解释策略易于部署，并且无需在生产环境中维护HTE模型。 摘要：Internet companies are increasingly using machine learning models to create personalized policies which assign, for each individual, the best predicted treatment for that individual. They are frequently derived from black-box heterogeneous treatment effect (HTE) models that predict individual-level treatment effects. In this paper, we focus on (1) learning explanations for HTE models; (2) learning interpretable policies that prescribe treatment assignments. We also propose guidance trees, an approach to ensemble multiple interpretable policies without the loss of interpretability. These rule-based interpretable policies are easy to deploy and avoid the need to maintain a HTE model in a production environment.
【3】 An overview of event extraction and its applications 标题：事件抽取及其应用综述 链接：https://arxiv.org/abs/2111.03212
作者：Jiangwei Liu,Liangyu Min,Xiaohong Huang 机构：School of Information Management and Engineering, Shanghai University of Finance and Economics, Shanghai , China 摘要：随着信息技术的飞速发展，网络平台产生了巨大的文本资源。作为一种特殊的信息抽取形式，事件抽取（EE）由于能够自动从人类语言中提取事件而越来越受欢迎。然而，关于事件提取的文献综述有限。现有的评审工作要么花费大量精力描述各种方法的细节，要么专注于特定领域。本研究全面概述了最先进的文本事件提取方法及其应用，包括封闭域和开放域事件提取。这项调查的一个特点是，它提供了一个中等复杂度的概述，避免涉及太多特定方法的细节。本研究侧重于讨论代表作品的共同特点、应用领域、优缺点，而忽略了个别方法的特殊性。最后，我们总结了常见的问题、当前的解决方案以及未来的研究方向。我们希望这项工作能够帮助研究人员和实践者快速了解最近的事件提取。 摘要：With the rapid development of information technology, online platforms have produced enormous text resources. As a particular form of Information Extraction (IE), Event Extraction (EE) has gained increasing popularity due to its ability to automatically extract events from human language. However, there are limited literature surveys on event extraction. Existing review works either spend much effort describing the details of various approaches or focus on a particular field. This study provides a comprehensive overview of the state-of-the-art event extraction methods and their applications from text, including closed-domain and open-domain event extraction. A trait of this survey is that it provides an overview in moderate complexity, avoiding involving too many details of particular approaches. This study focuses on discussing the common characters, application fields, advantages, and disadvantages of representative works, ignoring the specificities of individual approaches. Finally, we summarize the common issues, current solutions, and future research directions. We hope this work could help researchers and practitioners obtain a quick overview of recent event extraction.
推荐(1篇)
【1】 FINN.no Slates Dataset: A new Sequential Dataset Logging Interactions, allViewed Items and Click Responses/No-Click for Recommender Systems Research 标题：Finn.no Slate DataSet：一种新的顺序数据集，记录交互、所有查看的项目和点击响应/无点击以进行推荐系统研究 链接：https://arxiv.org/abs/2111.03340
作者：Simen Eide,Arnoldo Frigessi,Helge Jenssen,David S. Leslie,Joakim Rishaug,Sofie Verrewaere 机构：CCS Concepts: • Information systems → Personalization., Additional Key Words and Phrases: slate recommendations, search result, candidate sampling, marketplace data, reinforcement, learning, bandit, item attributes, off-policy, ACM Reference Format: 备注：5 pages, Fifteen ACM Conference on Recommender Systems (recsys21), 2021, Amsterdam, Netherlands 摘要：我们提出了一个新的推荐系统数据集，记录用户和在线市场之间的顺序交互。用户将依次获得来自市场的推荐和搜索结果，这些推荐和搜索结果以项目列表的形式排列，称为slates。数据集包括在每一轮中显示的板岩，用户是否单击了这些项目中的任何一个，以及用户单击了哪些项目。尽管推荐系统中暴露数据的使用正在增长，但据我们所知，没有一个开放的大规模推荐系统数据集包含在每次交互中呈现给用户的项目列表。因此，大多数关于推荐系统的文章都没有利用这种曝光信息。相反，所提出的模型只依赖于用户的点击响应，并假设用户在每一步都暴露于项目范围中的所有项目，通常称为统一候选抽样。这是一个不完整的假设，因为它考虑了用户可能没有接触过的项目。这样，用户可能会错误地认为项目不感兴趣。考虑到实际显示的板岩，模型可以使用更自然的可能性，基于给定项目暴露集的点击概率，这在bandit和强化学习文献中很普遍引用{Eide2021DynamicSampling}表明基于统一候选抽样（和类似假设）的可能性隐含地假设平台仅向用户显示最相关的项目。这会导致推荐系统隐式地加强反馈循环，并偏向于先前向用户公开的项目。 摘要：We present a novel recommender systems dataset that records the sequential interactions between users and an online marketplace. The users are sequentially presented with both recommendations and search results in the form of ranked lists of items, called slates, from the marketplace. The dataset includes the presented slates at each round, whether the user clicked on any of these items and which item the user clicked on. Although the usage of exposure data in recommender systems is growing, to our knowledge there is no open large-scale recommender systems dataset that includes the slates of items presented to the users at each interaction. As a result, most articles on recommender systems do not utilize this exposure information. Instead, the proposed models only depend on the user's click responses, and assume that the user is exposed to all the items in the item universe at each step, often called uniform candidate sampling. This is an incomplete assumption, as it takes into account items the user might not have been exposed to. This way items might be incorrectly considered as not of interest to the user. Taking into account the actually shown slates allows the models to use a more natural likelihood, based on the click probability given the exposure set of items, as is prevalent in the bandit and reinforcement learning literature. cite{Eide2021DynamicSampling} shows that likelihoods based on uniform candidate sampling (and similar assumptions) are implicitly assuming that the platform only shows the most relevant items to the user. This causes the recommender system to implicitly reinforce feedback loops and to be biased towards previously exposed items to the user.
聚类(2篇)
【1】 An Analysis of Elephants' Movement Data in Sub-Saharan Africa Using Clustering 标题：基于聚类的撒哈拉以南非洲大象迁徙数据分析 链接：https://arxiv.org/abs/2111.03533
作者：Gregory Glatzer,Prasenjit Mitra,Johnson Kinyua 机构：College of Information Science and Technology, The Pennsylvania State University, United States 备注：Presented at the 13th Annual TAWIRI Scientific Conference 摘要：了解动物的活动对保护工作至关重要。过去的研究通常关注影响运动的因素，而不是动物返回的兴趣地点或栖息地。我们探索使用聚类来确定撒哈拉以南非洲地区非洲象感兴趣的位置。我们的分析是使用南非克鲁格国家公园（KNP）的追踪非洲大象的公开数据集进行的；纳米比亚埃托沙国家公园；以及布基纳法索和刚果的一些地区。使用DBSCAN和KMeans聚类算法，我们计算聚类和质心，以简化大象移动数据并突出重要的兴趣位置。通过比较有温度和无温度的特征空间，我们发现温度是解释运动聚类的一个重要特征。认识到温度的重要性，我们开发了一种技术，将API中的外部温度数据添加到其他没有温度数据的地理空间数据集中。在解决了使用外部数据具有不同的时间戳的障碍之后，我们考虑了该数据的质量，以及基于该外部温度数据计算的簇的质心的质量。最后，我们将这些质心叠加到卫星图像和人类住区的位置上，以验证计算出的质心在识别大象感兴趣的位置方面的实际应用。正如预期的那样，我们确认大象倾向于聚集在水源和一些人类住区周围，特别是那些有水洞的地方。确定大象感兴趣的关键地点有助于预测大象的移动和防止偷猎。这些方法将来可能应用于大象以外的其他动物，以确定它们感兴趣的位置。 摘要：Understanding the movement of animals is crucial to conservation efforts. Past research often focuses on factors affecting movement, rather than locations of interest that animals return to or habitat. We explore the use of clustering to identify locations of interest to African Elephants in regions of Sub-Saharan Africa. Our analysis was performed using publicly available datasets for tracking African elephants at Kruger National Park (KNP), South Africa; Etosha National Park, Namibia; as well as areas in Burkina Faso and the Congo. Using the DBSCAN and KMeans clustering algorithms, we calculate clusters and centroids to simplify elephant movement data and highlight important locations of interest. Through a comparison of feature spaces with and without temperature, we show that temperature is an important feature to explain movement clustering. Recognizing the importance of temperature, we develop a technique to add external temperature data from an API to other geospatial datasets that would otherwise not have temperature data. After addressing the hurdles of using external data with marginally different timestamps, we consider the quality of this data, and the quality of the centroids of the clusters calculated based on this external temperature data. Finally, we overlay these centroids onto satellite imagery and locations of human settlements to validate the real-life applications of the calculated centroids to identify locations of interest for elephants. As expected, we confirmed that elephants tend to cluster their movement around sources of water as well as some human settlements, especially those with water holes. Identifying key locations of interest for elephants is beneficial in predicting the movement of elephants and preventing poaching. These methods may in the future be applied to other animals beyond elephants to identify locations of interests for them.
【2】 ExClus: Explainable Clustering on Low-dimensional Data Representations 标题：Exclus：低维数据表示上的可解释聚类 链接：https://arxiv.org/abs/2111.03168
作者：Xander Vankwikelberge,Bo Kang,Edith Heiter,Jefrey Lijffijt 机构：Ghent University, Ghent, Belgium 备注：15 pages, 7 figures 摘要：降维和聚类技术常用于分析复杂数据集，但其结果往往不易解释。我们考虑如何支持用户解释在不可直接解释的散点图上的明显的簇结构，例如当数据使用降维方法投射到二维空间时。具体来说，我们提出了一种自动计算可解释聚类的新方法，其中解释在原始高维空间中，聚类在低维投影中是一致的。它通过使用信息理论，在复杂性和提供的信息量之间提供了一个可调整的平衡。我们研究了这个问题的计算复杂性，并引入了对解的搜索空间的限制，以得到一个高效、可调、贪婪的优化算法。该算法进一步在一个名为ExClus的交互式工具中实现。在多个数据集上的实验突出表明，ExClus可以提供信息丰富且易于理解的模式，并且它们揭示了算法的有效性，以及考虑到可调性和可伸缩性的改进空间。 摘要：Dimensionality reduction and clustering techniques are frequently used to analyze complex data sets, but their results are often not easy to interpret. We consider how to support users in interpreting apparent cluster structure on scatter plots where the axes are not directly interpretable, such as when the data is projected onto a two-dimensional space using a dimensionality-reduction method. Specifically, we propose a new method to compute an interpretable clustering automatically, where the explanation is in the original high-dimensional space and the clustering is coherent in the low-dimensional projection. It provides a tunable balance between the complexity and the amount of information provided, through the use of information theory. We study the computational complexity of this problem and introduce restrictions on the search space of solutions to arrive at an efficient, tunable, greedy optimization algorithm. This algorithm is furthermore implemented in an interactive tool called ExClus. Experiments on several data sets highlight that ExClus can provide informative and easy-to-understand patterns, and they expose where the algorithm is efficient and where there is room for improvement considering tunability and scalability.
超分辨率|去噪|去模糊|去雾(1篇)
【1】 Remote Sensing Image Super-resolution and Object Detection: Benchmark and State of the Art 标题：遥感图像超分辨率与目标检测：基准与现状 链接：https://arxiv.org/abs/2111.03260
作者：Yi Wang,Syed Muhammad Arsalan Bashir,Mahrukh Khan,Qudrat Ullah,Rui Wang,Yilin Song,Zhe Guo,Yilong Niu 备注：39 pages, 15 figures, 5 tables. Submitted to Elsevier journal for review 摘要：在过去的二十年中，人们一直在努力开发遥感图像中的目标检测方法。在大多数情况下，遥感图像中用于小目标检测的数据集不足。许多研究人员使用场景分类数据集进行目标检测，这有其局限性；例如，在对象类别中，大型对象的数量超过小型对象。因此，它们缺乏多样性；这进一步影响了遥感图像中小目标检测器的检测性能。本文回顾了当前遥感图像的数据集和目标检测方法（基于深度学习）。我们还提出了一个大规模、公开的基准遥感超分辨率目标检测（RSSOD）数据集。RSSOD数据集由1759幅手工注释图像和22091幅空间分辨率为~0.05 m的甚高分辨率（VHR）图像组成。有五个类别，每个类别的标签频率不同。从卫星图像中提取图像块，包括真实图像畸变，如切向尺度畸变和倾斜畸变。我们还提出了一种新的多类循环超分辨率生成对抗网络（MCGR）和辅助YOLOv5检测器，对基于图像超分辨率的目标检测进行基准测试，并与现有的基于图像超分辨率（SR）的方法进行了比较。与目前最先进的NLSN方法相比，所提出的MCGR实现了图像SR的最先进性能，PSNR提高了1.2dB。对于五类、四类、两类和单类，MCGR分别获得了0.758、0.881、0.841和0.983的最佳目标检测图，分别超过了最先进的目标检测器YOLOv5、EfficientSet、更快的RCNN、SSD和RetinaNet的性能。 摘要：For the past two decades, there have been significant efforts to develop methods for object detection in Remote Sensing (RS) images. In most cases, the datasets for small object detection in remote sensing images are inadequate. Many researchers used scene classification datasets for object detection, which has its limitations; for example, the large-sized objects outnumber the small objects in object categories. Thus, they lack diversity; this further affects the detection performance of small object detectors in RS images. This paper reviews current datasets and object detection methods (deep learning-based) for remote sensing images. We also propose a large-scale, publicly available benchmark Remote Sensing Super-resolution Object Detection (RSSOD) dataset. The RSSOD dataset consists of 1,759 hand-annotated images with 22,091 instances of very high resolution (VHR) images with a spatial resolution of ~0.05 m. There are five classes with varying frequencies of labels per class. The image patches are extracted from satellite images, including real image distortions such as tangential scale distortion and skew distortion. We also propose a novel Multi-class Cyclic super-resolution Generative adversarial network with Residual feature aggregation (MCGR) and auxiliary YOLOv5 detector to benchmark image super-resolution-based object detection and compare with the existing state-of-the-art methods based on image super-resolution (SR). The proposed MCGR achieved state-of-the-art performance for image SR with an improvement of 1.2dB PSNR compared to the current state-of-the-art NLSN method. MCGR achieved best object detection mAPs of 0.758, 0.881, 0.841, and 0.983, respectively, for five-class, four-class, two-class, and single classes, respectively surpassing the performance of the state-of-the-art object detectors YOLOv5, EfficientDet, Faster RCNN, SSD, and RetinaNet.
自动驾驶|车辆|车道检测等(1篇)
【1】 Compressing Sensor Data for Remote Assistance of Autonomous Vehicles using Deep Generative Models 标题：基于深度产生式模型的自主车辆远程辅助传感器数据压缩 链接：https://arxiv.org/abs/2111.03201
作者：Daniel Bogdoll,Johannes Jestram,Jonas Rauch,Christin Scheib,Moritz Wittig,J. Marius Zöllner 机构：de 2KarlsruheInstituteofTechnology 备注：Daniel Bogdoll, Johannes Jestram, Jonas Rauch and Christin Scheib contributed equally. Accepted for publication at NeurIPS 2021 ML4AD Workshop 摘要：在可预见的未来，自动驾驶车辆在无法自行解决的情况下将需要人工协助。在这种情况下，来自人的远程协助可以为车辆提供继续运行所需的输入。自动驾驶车辆中使用的典型传感器包括摄像头和激光雷达传感器。由于必须实时发送大量传感器数据，高效的数据压缩对于防止网络基础设施过载至关重要。对于图像和激光雷达数据，使用深度生成神经网络的传感器数据压缩在压缩率和重建质量方面都优于传统的压缩方法。然而，对于基于生成神经网络的远程协助压缩算法的性能，目前还缺乏研究。为了深入了解用于远程协助的深层生成模型的可行性，我们评估了最先进的算法的适用性，并确定了潜在的弱点。此外，我们实现了用于处理传感器数据的在线管道，并使用CARLA模拟器演示了其远程协助性能。 摘要：In the foreseeable future, autonomous vehicles will require human assistance in situations they can not resolve on their own. In such scenarios, remote assistance from a human can provide the required input for the vehicle to continue its operation. Typical sensors used in autonomous vehicles include camera and lidar sensors. Due to the massive volume of sensor data that must be sent in real-time, highly efficient data compression is elementary to prevent an overload of network infrastructure. Sensor data compression using deep generative neural networks has been shown to outperform traditional compression approaches for both image and lidar data, regarding compression rate as well as reconstruction quality. However, there is a lack of research about the performance of generative-neural-network-based compression algorithms for remote assistance. In order to gain insights into the feasibility of deep generative models for usage in remote assistance, we evaluate state-of-the-art algorithms regarding their applicability and identify potential weaknesses. Further, we implement an online pipeline for processing sensor data and demonstrate its performance for remote assistance using the CARLA simulator.
点云|SLAM|雷达|激光|深度RGBD相关(1篇)
【1】 Interpreting Representation Quality of DNNs for 3D Point Cloud Processing 标题：三维点云处理中DNNs表示质量的解释 链接：https://arxiv.org/abs/2111.03549
作者：Wen Shen,Qihan Ren,Dongrui Liu,Quanshi Zhang 机构：Shanghai Jiao Tong University, Tongji University 摘要：在本文中，我们评估了三维点云处理中深层神经网络（DNN）编码的知识表示的质量。我们提出了一种将整体模型脆弱性分解为对旋转、平移、缩放和局部3D结构的敏感性的方法。此外，我们还提出了评估编码3D结构的空间平滑度和DNN表示复杂性的指标。基于这样的分析，实验揭示了经典DNN的表征问题，并解释了对抗训练的效用。 摘要：In this paper, we evaluate the quality of knowledge representations encoded in deep neural networks (DNNs) for 3D point cloud processing. We propose a method to disentangle the overall model vulnerability into the sensitivity to the rotation, the translation, the scale, and local 3D structures. Besides, we also propose metrics to evaluate the spatial smoothness of encoding 3D structures, and the representation complexity of the DNN. Based on such analysis, experiments expose representation problems with classic DNNs, and explain the utility of the adversarial training.
联邦学习|隐私保护|加密(3篇)
【1】 Data Selection for Efficient Model Update in Federated Learning 标题：联合学习中高效模型更新的数据选择 链接：https://arxiv.org/abs/2111.03512
作者：Hongrui Shi,Valentin Radu 机构：University of Sheffield 摘要：使用分布式数据训练集中式模型的联邦学习工作流越来越流行。然而，直到最近，这还是一个为客户提供类似计算能力的领域。快速扩展的物联网空间以及在边缘生成和处理的数据正在鼓励更多的努力扩展联邦学习，以包括异构系统。以前的方法将较小的模型分发给客户端，以提取本地数据的特征。但在客户端使用大量本地数据进行训练的问题仍然存在。我们建议减少训练全局模型所需的本地数据量。为此，我们将模型分为一个较低的部分进行一般特征提取，而上部对局部数据的特征更为敏感。我们通过对局部数据进行聚类并仅选择最具代表性的样本用于训练，减少了训练上半部分所需的数据量。我们的实验表明，使用我们的狭缝网络方法，只有不到1%的本地数据可以将客户数据的特征传输到全局模型。这些初步结果鼓励继续进行联合学习，减少计算资源有限的设备上的数据量，但这些设备拥有关键信息，有助于建立全球模型。 摘要：The Federated Learning workflow of training a centralized model with distributed data is growing in popularity. However, until recently, this was the realm of contributing clients with similar computing capabilities. The fast expanding IoT space and data being generated and processed at the edge are encouraging more effort into expanding federated learning to include heterogeneous systems. Previous approaches distribute smaller models to clients for distilling the characteristic of local data. But the problem of training with vast amounts of local data on the client side still remains. We propose to reduce the amount of local data that is needed to train a global model. We do this by splitting the model into a lower part for generic feature extraction and an upper part that is more sensitive to the characteristics of the local data. We reduce the amount of data needed to train the upper part by clustering the local data and selecting only the most representative samples to use for training. Our experiments show that less than 1% of the local data can transfer the characteristics of the client data to the global model with our slit network approach. These preliminary results are encouraging continuing towards federated learning with reduced amount of data on devices with limited computing resources, but which hold critical information to contribute to the global model.
【2】 FedLess: Secure and Scalable Federated Learning Using Serverless Computing 标题：FedLess：使用无服务器计算的安全、可扩展的联邦学习 链接：https://arxiv.org/abs/2111.03396
作者：Andreas Grafberger,Mohak Chadha,Anshul Jindal,Jianfeng Gu,Michael Gerndt 机构：∗Chair of Computer Architecture and Parallel Systems, Technische Universit¨at M¨unchen, Garching (near Munich), Germany 备注：IEEE BigData 2021 摘要：传统的以云为中心的深度学习（DL）方法要求在中央服务器上收集和处理训练数据，这在医疗保健等隐私敏感领域往往是一个挑战。为此，提出了一种称为联邦学习（FL）的新学习范式，它在解决隐私和数据所有权问题的同时，将DL的潜力引入这些领域。FL使远程客户端能够在保持数据本地的同时学习共享ML模型。然而，传统的FL系统面临一些挑战，如可扩展性、复杂的基础设施管理以及由于空闲客户端而造成的计算浪费和成本增加。FL系统的这些挑战与无服务器计算和功能即服务（FaaS）平台旨在解决的核心问题密切相关。其中包括快速可扩展性、无基础设施管理、空闲客户端自动扩展到零，以及按使用付费计费模式。为此，我们提出了一种新的无服务器FL系统和框架，称为FedLess。我们的系统支持多个商业和自托管FaaS提供商，可以部署在云中、机构数据中心内部部署和边缘设备上。据我们所知，我们是第一个跨大量异构FaaS提供商实现FL的公司，同时提供安全和差异隐私等重要功能。我们通过全面的实验证明，使用我们的系统，针对多达200个客户机功能和更多功能的不同任务成功训练DNN是可能的。此外，通过与传统FL系统的比较，我们证明了我们的方法的实际可行性，并表明它可以更便宜、更节省资源。 摘要：The traditional cloud-centric approach for Deep Learning (DL) requires training data to be collected and processed at a central server which is often challenging in privacy-sensitive domains like healthcare. Towards this, a new learning paradigm called Federated Learning (FL) has been proposed that brings the potential of DL to these domains while addressing privacy and data ownership issues. FL enables remote clients to learn a shared ML model while keeping the data local. However, conventional FL systems face several challenges such as scalability, complex infrastructure management, and wasted compute and incurred costs due to idle clients. These challenges of FL systems closely align with the core problems that serverless computing and Function-as-a-Service (FaaS) platforms aim to solve. These include rapid scalability, no infrastructure management, automatic scaling to zero for idle clients, and a pay-per-use billing model. To this end, we present a novel system and framework for serverless FL, called FedLess. Our system supports multiple commercial and self-hosted FaaS providers and can be deployed in the cloud, on-premise in institutional data centers, and on edge devices. To the best of our knowledge, we are the first to enable FL across a large fabric of heterogeneous FaaS providers while providing important features like security and Differential Privacy. We demonstrate with comprehensive experiments that the successful training of DNNs for different tasks across up to 200 client functions and more is easily possible using our system. Furthermore, we demonstrate the practical viability of our methodology by comparing it against a traditional FL system and show that it can be cheaper and more resource-efficient.
【3】 DVFL: A Vertical Federated Learning Method for Dynamic Data 标题：DVFL：一种面向动态数据的垂直联合学习方法 链接：https://arxiv.org/abs/2111.03341
作者：Yuzhi Liang,Yixiang Chen 机构： Peking Univerisity Shen-zhen Graduate School 摘要：联邦学习通过将多个计算设备连接到一个分散的系统中来解决数据孤岛问题，已成为一种很有前途的隐私保护机器学习模式。本文研究了垂直联合学习（VFL），它解决了协作组织共享同一组用户但不相交的特征的情况。当代VFL方法主要用于静态场景，其中主动方和被动方从一开始就拥有所有数据，并且不会改变。然而，现实生活中的数据往往是动态变化的。为了缓解这个问题，我们提出了一种新的垂直联邦学习方法DVFL，该方法通过知识提取来适应动态数据分布的变化。在DVFL中，大多数计算都在本地进行，以提高数据安全性和模型效率。我们的大量实验结果表明，DVFL不仅可以在静态场景中获得接近现有VFL方法的结果，而且可以适应动态场景中数据分布的变化。 摘要：Federated learning, which solves the problem of data island by connecting multiple computational devices into a decentralized system, has become a promising paradigm for privacy-preserving machine learning. This paper studies vertical federated learning (VFL), which tackles the scenarios where collaborating organizations share the same set of users but disjoint features. Contemporary VFL methods are mainly used in static scenarios where the active party and the passive party have all the data from the beginning and will not change. However, the data in real life often changes dynamically. To alleviate this problem, we propose a new vertical federation learning method, DVFL, which adapts to dynamic data distribution changes through knowledge distillation. In DVFL, most of the computations are held locally to improve data security and model efficiency. Our extensive experimental results show that DVFL can not only obtain results close to existing VFL methods in static scenes, but also adapt to changes in data distribution in dynamic scenarios.
推理|分析|理解|解释(7篇)
【1】 Investigation of Topic Modelling Methods for Understanding the Reports of the Mining Projects in Queensland 标题：昆士兰矿业项目报告理解的主题建模方法研究 链接：https://arxiv.org/abs/2111.03576
作者：Yasuko Okamoto,Thirunavukarasu Balasubramaniam,Richi Nayak 机构： RecordPoint, Australia., School of Computer Science and Centre for Data Science, Queensland University of, Technology, Brisbane, Australia. 备注：Accepted in The 19th Australasian Data Mining Conference 2021 摘要：在采矿行业，许多报告是在项目管理过程中生成的。这些过去的文档是未来成功的巨大知识资源。但是，如果文档是无组织和非结构化的，那么检索必要的信息将是一项繁琐而具有挑战性的任务。文档聚类是解决这一问题的有效方法，在过去的研究中已经引入了许多方法。尽管如此，对于任何类型的文档，都没有能够发挥最佳性能的银弹。因此，需要对新数据集应用聚类方法进行探索性研究。在本研究中，我们将研究多主题建模（TM）方法。目标是使用昆士兰政府资源部昆士兰地质调查局的数据集，找到采矿项目报告的适当方法，并了解内容，了解如何组织这些报告。对三种TM方法：潜在Dirichlet分配（LDA）、非负矩阵分解（NMF）和非负张量分解（NTF）进行了统计和定性比较。经过评估，我们得出结论：LDA对数据集的性能最好；然而，仍有可能采用其他方法，并加以改进。 摘要：In the mining industry, many reports are generated in the project management process. These past documents are a great resource of knowledge for future success. However, it would be a tedious and challenging task to retrieve the necessary information if the documents are unorganized and unstructured. Document clustering is a powerful approach to cope with the problem, and many methods have been introduced in past studies. Nonetheless, there is no silver bullet that can perform the best for any types of documents. Thus, exploratory studies are required to apply the clustering methods for new datasets. In this study, we will investigate multiple topic modelling (TM) methods. The objectives are finding the appropriate approach for the mining project reports using the dataset of the Geological Survey of Queensland, Department of Resources, Queensland Government, and understanding the contents to get the idea of how to organise them. Three TM methods, Latent Dirichlet Allocation (LDA), Nonnegative Matrix Factorization (NMF), and Nonnegative Tensor Factorization (NTF) are compared statistically and qualitatively. After the evaluation, we conclude that the LDA performs the best for the dataset; however, the possibility remains that the other methods could be adopted with some improvements.
【2】 Epidemic inference through generative neural networks 标题：基于产生式神经网络的流行病推理 链接：https://arxiv.org/abs/2111.03383
作者：Indaco Biazzo,Alfredo Braunstein,Luca Dall'Asta,Fabio Mazza 机构： Politecnico di Torino, Corso Duca degli Abruzzi , Torino, Italy, IIGM - Italian Institute for Genomic Medicine, co IRCSS, Candiolo, Torino, Italy, Collegio Carlo Alberto, P.za Arbarello , Torino, Italy 备注：30 pages, 6 figures, 1 table 摘要：在传染病在接触网上传播过程中，重建缺失的信息对于预防和遏制战略至关重要。例如，识别和警告感染但无症状的个体（例如，手动接触追踪）有助于遏制新冠病毒-19大流行的爆发。可能的流行病级联的数量通常随着涉及的个体数量呈指数增长。流行病过程中的推理问题所带来的挑战源于难以确定那些与证据相符的几乎可以忽略不计的子集（例如，医学测试）。在这里，我们提出了一种新的生成性神经网络框架，它可以对最可能的感染级联进行采样，并与观察结果相一致。此外，该框架还可以推断控制感染传播的参数。该方法在患者零问题、风险评估以及在合成和真实病例场景（如工作场所和医院的传染病传播）中的传染参数推断方面，与现有方法相比，取得了更好或可比的结果。 摘要：Reconstructing missing information in epidemic spreading on contact networks can be essential in prevention and containment strategies. For instance, identifying and warning infective but asymptomatic individuals (e.g., manual contact tracing) helped contain outbreaks in the COVID-19 pandemic. The number of possible epidemic cascades typically grows exponentially with the number of individuals involved. The challenge posed by inference problems in the epidemics processes originates from the difficulty of identifying the almost negligible subset of those compatible with the evidence (for instance, medical tests). Here we present a new generative neural networks framework that can sample the most probable infection cascades compatible with observations. Moreover, the framework can infer the parameters governing the spreading of infections. The proposed method obtains better or comparable results with existing methods on the patient zero problem, risk assessment, and inference of infectious parameters in synthetic and real case scenarios like spreading infections in workplaces and hospitals.
【3】 Analysis of Sensing Spectral for Signal Recovery Under a Generalized Linear Model 标题：广义线性模型下信号恢复的传感谱分析 链接：https://arxiv.org/abs/2111.03237
作者：Junjie Ma,Ji Xu,Arian Maleki 备注：accepted by Neurips 2021 摘要：我们考虑一个非线性逆问题$MaTHBF {y}= f（ Mathbf {Ax}）$，其中$MaTHBB{r} m $中的$$MATHBF {Y}是$ MATHBF {AX}在 MaTHBB{R} m $中的分量非线性变换，$ Mththbf{x}在 MthBB{R} n $中是感兴趣的信号，$ MathBF {A}$是已知的线性映射。通过适当地指定非线性处理函数，该模型可以具体化为许多信号处理问题，包括压缩感知和相位恢复。本文的主要目的是了解感知矩阵，或者更具体地说，感知矩阵的频谱，对从$mathbf{y}中恢复$mathbf{x}$的难度的影响。为此，我们研究了最成功的恢复方法之一，即期望传播算法（EP）的性能。我们定义了$mathbf{a}$谱尖峰的概念，并说明了该度量在EP性能中的重要性。频谱的尖峰是否会损害或帮助EP的恢复性能取决于$f$。我们根据函数$f$定义了某些量，该函数使我们能够描述光谱尖峰对EP恢复的影响。基于我们的框架，我们能够证明，例如，在相位恢复问题中，具有尖峰频谱的矩阵更适合EP，而在1位压缩感知问题中，尖峰（较平坦）频谱的恢复效果更好。我们的结果统一并实质上推广了现有的比较次高斯矩阵和正交矩阵的结果，并为设计最佳传感系统提供了一个平台。 摘要：We consider a nonlinear inverse problem $mathbf{y}= f(mathbf{Ax})$, where observations $mathbf{y} in mathbb{R}^m$ are the componentwise nonlinear transformation of $mathbf{Ax} in mathbb{R}^m$, $mathbf{x} in mathbb{R}^n$ is the signal of interest and $mathbf{A}$ is a known linear mapping. By properly specifying the nonlinear processing function, this model can be particularized to many signal processing problems, including compressed sensing and phase retrieval. Our main goal in this paper is to understand the impact of sensing matrices, or more specifically the spectrum of sensing matrices, on the difficulty of recovering $mathbf{x}$ from $mathbf{y}$. Towards this goal, we study the performance of one of the most successful recovery methods, i.e. the expectation propagation algorithm (EP). We define a notion for the spikiness of the spectrum of $mathbf{A}$ and show the importance of this measure in the performance of the EP. Whether the spikiness of the spectrum can hurt or help the recovery performance of EP depends on $f$. We define certain quantities based on the function $f$ that enables us to describe the impact of the spikiness of the spectrum on EP recovery. Based on our framework, we are able to show that for instance, in phase-retrieval problems, matrices with spikier spectrums are better for EP, while in 1-bit compressed sensing problems, less spiky (flatter) spectrums offer better recoveries. Our results unify and substantially generalize the existing results that compare sub-Gaussian and orthogonal matrices, and provide a platform toward designing optimal sensing systems.
【4】 Explainable k-means. Don't be greedy, plant bigger trees! 标题：可解释的k-均值。别贪心了，种更大的树吧！ 链接：https://arxiv.org/abs/2111.03193
作者：Konstantin Makarychev,Liren Shan 机构：Northwestern University, Evanston, IL 备注：25 pages, 4 figures 摘要：我们提供了一种新的bi准则$tilde{O}（log^2k）$竞争算法，用于解释$k$-均值聚类。Dasgupta、Frost、Moshkovitz和Rashtchian最近引入了可解释的$k$-方法（ICML 2020）。它由一个易于解释和理解（基本要求）的决策树或图来描述。可解释的$k$-意味着集群的成本等于其集群的成本之和；每个簇的代价等于从簇中的点到簇中心的距离平方和。我们的随机双标准算法构造了一个阈值决策树，将数据集划分为$（1 delta）k$簇（其中$deltain（0,1）$是算法的一个参数）。此群集的成本最多为$tilde{O}（1/deltacdotlog^2k）$乘以最佳无约束$k$-均值群集的成本。我们证明了这个界几乎是最优的。 摘要：We provide a new bi-criteria $tilde{O}(log^2 k)$ competitive algorithm for explainable $k$-means clustering. Explainable $k$-means was recently introduced by Dasgupta, Frost, Moshkovitz, and Rashtchian (ICML 2020). It is described by an easy to interpret and understand (threshold) decision tree or diagram. The cost of the explainable $k$-means clustering equals to the sum of costs of its clusters; and the cost of each cluster equals the sum of squared distances from the points in the cluster to the center of that cluster. Our randomized bi-criteria algorithm constructs a threshold decision tree that partitions the data set into $(1 delta)k$ clusters (where $deltain (0,1)$ is a parameter of the algorithm). The cost of this clustering is at most $tilde{O}(1/delta cdot log^2 k)$ times the cost of the optimal unconstrained $k$-means clustering. We show that this bound is almost optimal.
【5】 Value Function Spaces: Skill-Centric State Abstractions for Long-Horizon Reasoning 标题：值函数空间：面向长视野推理的以技能为中心的状态抽象 链接：https://arxiv.org/abs/2111.03189
作者：Dhruv Shah,Peng Xu,Yao Lu,Ted Xiao,Alexander Toshev,Sergey Levine,Brian Ichter 机构：Berkeley AI Research, UC Berkeley 摘要：强化学习可以训练有效执行复杂任务的策略。然而，对于长时间的任务，这些方法的性能会随着时间的推移而下降，通常需要对较低级别的技能进行推理和组合。分层强化学习旨在通过提供一系列作为动作抽象的低级技能来实现这一点。层次结构还可以通过抽象空间状态来进一步改进这一点。我们假定，合适的状态抽象应该取决于可用的较低级别策略的能力。我们提出了值函数空间：一种简单的方法，通过使用对应于每个较低级别技能的值函数来产生这种表示。这些值函数捕捉场景的启示，从而形成一个表示，该表示紧凑地抽象了任务相关信息，并稳健地忽略了干扰因素。对迷宫求解和机器人操作任务的实证评估表明，与其他无模型和基于模型的方法相比，我们的方法提高了长视野性能，并实现了更好的Zero-Shot泛化。 摘要：Reinforcement learning can train policies that effectively perform complex tasks. However for long-horizon tasks, the performance of these methods degrades with horizon, often necessitating reasoning over and composing lower-level skills. Hierarchical reinforcement learning aims to enable this by providing a bank of low-level skills as action abstractions. Hierarchies can further improve on this by abstracting the space states as well. We posit that a suitable state abstraction should depend on the capabilities of the available lower-level policies. We propose Value Function Spaces: a simple approach that produces such a representation by using the value functions corresponding to each lower-level skill. These value functions capture the affordances of the scene, thus forming a representation that compactly abstracts task relevant information and robustly ignores distractors. Empirical evaluations for maze-solving and robotic manipulation tasks demonstrate that our approach improves long-horizon performance and enables better zero-shot generalization than alternative model-free and model-based methods.
【6】 Amortized Variational Inference for Simple Hierarchical Models 标题：简单分层模型的分期变分推理 链接：https://arxiv.org/abs/2111.03144
作者：Abhinav Agrawal,Justin Domke 机构：College of Information and Computer Science, Univeristy Of Massachusetts Amherst 备注：Neural Information Processing Systems (NeurIPS) 2021 摘要：在层次模型中，由于局部潜在变量的数量随数据集的变化而变化，因此很难使用带有变分推理的子抽样。因此，分层模型中的推理在大规模上仍然是一个挑战。使用结构与后验分布相匹配的变分族是有帮助的，但由于局部分布数量巨大，优化速度仍然很慢。相反，本文提出了一种摊销方法，其中共享参数同时表示所有局部分布。这种方法与使用给定的联合分布（例如，满秩高斯分布）一样精确，但在大几个数量级的数据集上是可行的。它也比使用结构化变分分布快得多。 摘要：It is difficult to use subsampling with variational inference in hierarchical models since the number of local latent variables scales with the dataset. Thus, inference in hierarchical models remains a challenge at large scale. It is helpful to use a variational family with structure matching the posterior, but optimization is still slow due to the huge number of local distributions. Instead, this paper suggests an amortized approach where shared parameters simultaneously represent all local distributions. This approach is similarly accurate as using a given joint distribution (e.g., a full-rank Gaussian) but is feasible on datasets that are several orders of magnitude larger. It is also dramatically faster than using a structured variational distribution.
【7】 Pathological Analysis of Blood Cells Using Deep Learning Techniques 标题：基于深度学习技术的血细胞病理学分析 链接：https://arxiv.org/abs/2111.03274
作者：Virender Ranga,Shivam Gupta,Priyansh Agrawal,Jyoti Meena 机构：Department of Computer Engineering, National Institute of Technology , Kurukshetra , Haryana, India, Department of Computer Science and Engineering, Indian Institute of Information Technology , Sonepat ,Haryana , India 备注：None 摘要：病理学是通过分析身体样本来发现疾病原因的实践。该领域最常用的方法是使用组织学，基本上是研究和观察细胞和组织的微观结构。幻灯片查看方法被广泛使用，并转换为数字形式，以生成高分辨率图像。这使得深度学习和机器学习领域能够深入到医学领域。在本研究中，提出了一种基于神经网络的血细胞图像分类方法。当输入图像通过所提出的结构并按照所提出的算法使用所有超参数和丢失率值时，该模型对血液图像进行分类，准确率为95.24%。该模型的性能优于现有的标准体系结构和众多研究人员的工作。因此，该模型将使病理系统的开发成为可能，从而减少人为错误和实验室人员的日常负荷。这将反过来帮助病理学家更有效地开展工作。 摘要：Pathology deals with the practice of discovering the reasons for disease by analyzing the body samples. The most used way in this field, is to use histology which is basically studying and viewing microscopic structures of cell and tissues. The slide viewing method is widely being used and converted into digital form to produce high resolution images. This enabled the area of deep learning and machine learning to deep dive into this field of medical sciences. In the present study, a neural based network has been proposed for classification of blood cells images into various categories. When input image is passed through the proposed architecture and all the hyper parameters and dropout ratio values are used in accordance with proposed algorithm, then model classifies the blood images with an accuracy of 95.24%. The performance of proposed model is better than existing standard architectures and work done by various researchers. Thus model will enable development of pathological system which will reduce human errors and daily load on laboratory men. This will in turn help pathologists in carrying out their work more efficiently and effectively.
检测相关(6篇)
【1】 Monitoring geometrical properties of word embeddings for detecting the emergence of new topics 标题：监控词嵌入的几何属性以检测新主题的出现 链接：https://arxiv.org/abs/2111.03496
作者：Clément Christophe,Julien Velcin,Jairo Cugliari,Manel Boumghar,Philippe Suignard 机构：Université de Lyon, Lyon , UR ERIC, France, EDF R&D, Palaiseau, France 摘要：缓慢出现的主题检测是一项介于事件检测和语言进化之间的任务，前者是我们在短时间内聚合不同单词的行为，后者是我们监控它们的长期进化。在这项工作中，我们解决了早期发现缓慢出现的新主题的问题。为此，我们在单词层面收集弱信号的证据。我们建议监控嵌入空间中单词表示的行为，并使用其几何特性来描述主题的出现。由于这类任务通常很难进行评估，因此我们提出了一个定量评估框架。我们在新闻和科学文章的两个公共数据集上显示了优于最新方法的积极结果。 摘要：Slow emerging topic detection is a task between event detection, where we aggregate behaviors of different words on short period of time, and language evolution, where we monitor their long term evolution. In this work, we tackle the problem of early detection of slowly emerging new topics. To this end, we gather evidence of weak signals at the word level. We propose to monitor the behavior of words representation in an embedding space and use one of its geometrical properties to characterize the emergence of topics. As evaluation is typically hard for this kind of task, we present a framework for quantitative evaluation. We show positive results that outperform state-of-the-art methods on two public datasets of press and scientific articles.
【2】 Dataset of Fake News Detection and Fact Verification: A Survey 标题：假新闻检测与事实验证数据集研究综述 链接：https://arxiv.org/abs/2111.03299
作者：Taichi Murayama 机构： NARA Institute of Science and Technology 备注：33pages, 5 tables 摘要：假新闻的迅速增加对社会造成了巨大的危害，引发了许多与假新闻相关的研究，包括假新闻检测和事实验证技术的发展。这些研究的资源主要是从网络数据中获取的公共数据集。我们从三个角度对118个与假新闻研究相关的数据集进行了大规模调查：（1）假新闻检测，（2）事实验证，（3）其他任务；例如，假新闻分析和讽刺检测。我们还详细描述了它们的使用任务和特点。最后，我们强调了假新闻数据集建设面临的挑战以及应对这些挑战的一些研究机会。我们的调查通过帮助研究人员在不重新发明轮子的情况下找到合适的数据集来促进假新闻研究，从而提高假新闻研究的深度。 摘要：The rapid increase in fake news, which causes significant damage to society, triggers many fake news related studies, including the development of fake news detection and fact verification techniques. The resources for these studies are mainly available as public datasets taken from Web data. We surveyed 118 datasets related to fake news research on a large scale from three perspectives: (1) fake news detection, (2) fact verification, and (3) other tasks; for example, the analysis of fake news and satire detection. We also describe in detail their utilization tasks and their characteristics. Finally, we highlight the challenges in the fake news dataset construction and some research opportunities that address these challenges. Our survey facilitates fake news research by helping researchers find suitable datasets without reinventing the wheel, and thereby, improves fake news studies in depth.
【3】 Automated Human Mind Reading Using EEG Signals for Seizure Detection 标题：利用脑电信号进行癫痫检测的自动读心术 链接：https://arxiv.org/abs/2111.03270
作者：Virender Ranga,Shivam Gupta,Jyoti Meena,Priyansh Agrawal 机构：Automated Human Mind Reading Using EEG Signals for Seizure Detection, Department of Computer Engineering, Department of Computer Science and Engineering, Indian Institute of Information Technology, Sonepat, Haryana, India 备注：None 摘要：癫痫是全球最常见的神经系统疾病之一，出现于公元前4000年。如今，它正影响着大约5000万不同年龄的人。这种疾病的特点是反复发作。在过去的几十年里，随着医学科学和技术的进步，用于控制癫痫发作的治疗方法有了很大的改进。脑电图（EEG）是一种广泛用于监测大脑活动的技术，广泛用于癫痫发作区域的检测。它是在手术前进行的，也可以在手术时预测癫痫发作，这在神经刺激装置中很有用。但在大多数情况下，视觉检查是由神经学家完成的，目的是检测和分类疾病的模式，但这需要大量的领域前知识和经验。这一切反过来给神经外科医生带来压力，导致时间浪费，也降低了他们的准确性和效率。在信息技术领域需要一些自动化系统，比如在深度学习中使用神经网络，这可以帮助神经学家。本文提出了一个模型，其精度为98.33%，可用于自动化系统的开发。开发的系统将大大有助于神经学家的表现。 摘要：Epilepsy is one of the most occurring neurological disease globally emerged back in 4000 BC. It is affecting around 50 million people of all ages these days. The trait of this disease is recurrent seizures. In the past few decades, the treatments available for seizure control have improved a lot with the advancements in the field of medical science and technology. Electroencephalogram (EEG) is a widely used technique for monitoring the brain activity and widely popular for seizure region detection. It is performed before surgery and also to predict seizure at the time operation which is useful in neuro stimulation device. But in most of cases visual examination is done by neurologist in order to detect and classify patterns of the disease but this requires a lot of pre-domain knowledge and experience. This all in turns put a pressure on neurosurgeons and leads to time wastage and also reduce their accuracy and efficiency. There is a need of some automated systems in arena of information technology like use of neural networks in deep learning which can assist neurologists. In the present paper, a model is proposed to give an accuracy of 98.33% which can be used for development of automated systems. The developed system will significantly help neurologists in their performance.
【4】 Neural Network Based Epileptic EEG Detection and Classification 标题：基于神经网络的癫痫脑电信号检测与分类 链接：https://arxiv.org/abs/2111.03268
作者：Shivam Gupta,Jyoti Meena,O. P Gupta 机构：a Department of Computer Science and Engineering , Indian Institute of Information Technology, Sonepat, Haryana, India (Mentor National Institute of Technology, Kurukshetra, Haryana, India ) 备注：None 摘要：及时诊断对挽救癫痫患者的生命至关重要。在过去的几年里，有很多治疗癫痫的方法。这些治疗需要使用抗癫痫药物，但不能有效控制癫痫发作的频率。需要通过手术切除受影响区域。脑电图（EEG）是一种广泛用于监测大脑活动的技术，广泛用于癫痫发作区域的检测。它在手术前用于定位受影响区域。这个手动过程，使用脑电图图，是耗时的，需要深入的专业知识。本文提出了一种以文本一维向量形式保持脑电信号真实性的模型。该模型在波恩大学数据集上取得了最先进的性能，对所有五类脑电数据分类的平均灵敏度、特异性分别为81%和81.4%。二元分类的特异性和敏感性得分分别达到99.9%和99.5%，而不是其他研究人员使用的2D模型。因此，开发的系统将极大地帮助神经外科医生提高他们的表现。 摘要：Timely diagnosis is important for saving the life of epileptic patients. In past few years, a lot of treatments are available for epilepsy. These treatments require use of anti-seizure drugs but are not effective in controlling frequency of seizure. There is need of removal of an affected region using surgery. Electroencephalogram (EEG) is a widely used technique for monitoring the brain activity and widely popular for seizure region detection. It is used before surgery for locating affected region. This manual process, using EEG graphs, is time consuming and requires deep expertise. In the present paper, a model has been proposed that preserves the true nature of an EEG signal in form of textual one-dimensional vector. The proposed model achieves a state of art performance for Bonn University dataset giving an average sensitivity, specificity of 81% and 81.4% respectively for classification of EEG data among all five classes. Also for binary classification achieving 99.9%, 99.5% score value for specificity and sensitivity instead of 2D models used by other researchers. Thus, developed system will significantly help neurosurgeons in the increase of their performance.
【5】 An Empirical Study of the Effectiveness of an Ensemble of Stand-alone Sentiment Detection Tools for Software Engineering Datasets 标题：软件工程数据集独立情感检测工具集成有效性的实证研究 链接：https://arxiv.org/abs/2111.03196
作者：Gias Uddin,Yann-Gael Gueheneuc,Foutse Khomh,Chanchal K Roy 机构：YANN-GAËL GUÉHÉNEUC, Concordia University, Canada, CHANCHAL K. ROY, University of Saskatchewan, Canada 备注：None 摘要：软件工程（SE）中的情感分析已显示出分析和支持各种开发活动的前景。我们报告了一项实证研究的结果，该研究旨在确定通过结合独立SE特定情绪检测器的极性标签开发集成引擎的可行性。我们的研究分为两个阶段。在第一阶段，我们从Lin等人[31,32]最近发表的两篇论文中选择了五种SE特定情绪检测工具，他们首先报告了独立情绪检测器的负面结果，然后提出了一种改进的SE特定情绪检测器POME[31]。我们报告了17581个单元（句子/文件）的研究结果，这些单元来自六个目前可用的SE情绪基准。我们发现，在85-95%的情况下，现有工具可以相互补充，即一个是错误的，另一个是正确的。然而，这些工具中基于多数票的集合无法提高情感检测的准确性。我们开发了Sentisead，这是一个有监督的工具，它将极性标签和单词包作为特征结合起来。Sentisead将单个工具的性能（F1分数）提高了4%（比Senti4SD[5]）-100%（比POME[31]）。在第二阶段，我们使用预先训练的Transformer模型（PTM）比较和改进Sentisead基础设施。我们发现，Sentisead基础设施与RoBERTa是五个独立的基于规则的浅层学习SE特定工具的集合，由Lin等人[31,32]提供，六个数据集的F1得分最高为0.805，而独立RoBERTa的F1得分为0.801。 摘要：Sentiment analysis in software engineering (SE) has shown promise to analyze and support diverse development activities. We report the results of an empirical study that we conducted to determine the feasibility of developing an ensemble engine by combining the polarity labels of stand-alone SE-specific sentiment detectors. Our study has two phases. In the first phase, we pick five SE-specific sentiment detection tools from two recently published papers by Lin et al. [31, 32], who first reported negative results with standalone sentiment detectors and then proposed an improved SE-specific sentiment detector, POME [31]. We report the study results on 17,581 units (sentences/documents) coming from six currently available sentiment benchmarks for SE. We find that the existing tools can be complementary to each other in 85-95% of the cases, i.e., one is wrong, but another is right. However, a majority voting-based ensemble of those tools fails to improve the accuracy of sentiment detection. We develop Sentisead, a supervised tool by combining the polarity labels and bag of words as features. Sentisead improves the performance (F1-score) of the individual tools by 4% (over Senti4SD [5]) - 100% (over POME [31]). In a second phase, we compare and improve Sentisead infrastructure using Pre-trained Transformer Models (PTMs). We find that a Sentisead infrastructure with RoBERTa as the ensemble of the five stand-alone rule-based and shallow learning SE-specific tools from Lin et al. [31, 32] offers the best F1-score of 0.805 across the six datasets, while a stand-alone RoBERTa shows an F1-score of 0.801.
【6】 Community detection in censored hypergraph 标题：删失超图中的社区检测 链接：https://arxiv.org/abs/2111.03179
作者：Mingao Yuan,Bin Zhao,Xiaofeng Zhao 机构：School of Mathematics and Statistics, North China University of Water Resources and Electric Power, China, e-mail: 摘要：社区检测是指将网络的节点（图或hypergrah）聚集到组中的问题。各种算法可用于社区检测，所有这些方法都适用于未经审查的网络。在实践中，网络可能存在截尾（或缺失）值，并且截尾值对网络的结构特性具有不可忽略的影响。本文从信息论的角度研究了删失$m$-一致超图中的群体检测问题。我们推导了准确恢复群落结构的信息论阈值。此外，我们还提出了一种多项式时间算法来精确地将社区结构恢复到阈值。该算法由一个谱算法和一个细化步骤组成。同样有趣的是，研究没有细化的单一光谱算法是否达到阈值。为此，我们还研究了半定松弛算法并分析了其性能。 摘要：Community detection refers to the problem of clustering the nodes of a network (either graph or hypergrah) into groups. Various algorithms are available for community detection and all these methods apply to uncensored networks. In practice, a network may has censored (or missing) values and it is shown that censored values have non-negligible effect on the structural properties of a network. In this paper, we study community detection in censored $m$-uniform hypergraph from information-theoretic point of view. We derive the information-theoretic threshold for exact recovery of the community structure. Besides, we propose a polynomial-time algorithm to exactly recover the community structure up to the threshold. The proposed algorithm consists of a spectral algorithm plus a refinement step. It is also interesting to study whether a single spectral algorithm without refinement achieves the threshold. To this end, we also explore the semi-definite relaxation algorithm and analyze its performance.
分类|识别(2篇)
【1】 Sexism Identification in Tweets and Gabs using Deep Neural Networks 标题：基于深度神经网络的推文和Gabs中的性别歧视识别 链接：https://arxiv.org/abs/2111.03612
作者：Amikul Kalra,Arkaitz Zubiaga 机构：School of Electronic Engineering and Computer Science, Queen Mary Universiy of London 备注：8 pages 摘要：通过匿名化和可访问性，社交媒体平台促进了仇恨言论的扩散，促进了开发自动识别这些文本的方法的研究。本文利用长短时记忆（LSTMs）和卷积神经网络（CNN）等多种深层神经网络模型结构，探讨了文本中性别歧视的分类。这些网络与Transformers（BERT）和DistilBERT模型的双向编码器表示形式的转移学习结合使用，以及数据扩充，以对社交网络中的性别歧视识别（EXIST）中的推特和GAB数据集执行二进制和多类性别歧视分类IberLEF 2021年的任务。与竞争对手的模型相比，这些模型的性能最好，使用BERT和多滤波器CNN模型。数据扩充进一步改进了多类分类任务的这些结果。本文还探讨了模型所犯的错误，并讨论了由于标签的主观性和社交媒体中使用的自然语言的复杂性而导致的性别歧视自动分类的困难。 摘要：Through anonymisation and accessibility, social media platforms have facilitated the proliferation of hate speech, prompting increased research in developing automatic methods to identify these texts. This paper explores the classification of sexism in text using a variety of deep neural network model architectures such as Long-Short-Term Memory (LSTMs) and Convolutional Neural Networks (CNNs). These networks are used in conjunction with transfer learning in the form of Bidirectional Encoder Representations from Transformers (BERT) and DistilBERT models, along with data augmentation, to perform binary and multiclass sexism classification on the dataset of tweets and gabs from the sEXism Identification in Social neTworks (EXIST) task in IberLEF 2021. The models are seen to perform comparatively to those from the competition, with the best performances seen using BERT and a multi-filter CNN model. Data augmentation further improves these results for the multi-class classification task. This paper also explores the errors made by the models and discusses the difficulty in automatically classifying sexism due to the subjectivity of the labels and the complexity of natural language used in social media.
【2】 Application of Machine Learning to Sleep Stage Classification 标题：机器学习在睡眠阶段分类中的应用 链接：https://arxiv.org/abs/2111.03085
作者：Andrew Smith,Hardik Anand,Snezana Milosavljevic,Katherine M. Rentschler,Ana Pocivavsek,Homayoun Valafar 机构：Department of Computer Science and Engineering, (University of South Carolina), Columbia, SC , USA, Department of Pharmacology, Physiology, and Neuroscience, University of South Carolina School of Medicine 备注：6 pages, submitted to CSCI 2021 摘要：睡眠研究必须重述与睡眠丧失相关的表型，并揭示有助于精神病理学的机制。大多数情况下，研究人员手动将多导睡眠图分为警戒状态，这很耗时，需要大量训练，并且容易出现记分员之间的差异。虽然许多研究已经成功地开发了基于多个EEG通道的自动警戒状态分类器，但我们的目标是开发一种能够基于单个皮层脑电图（EEG）可靠预测警戒状态的自动开放访问分类器从啮齿类动物，以尽量减少的缺点，伴随着拴小动物通过电线到计算机程序。在571小时的总数据中，领域专家对大约427小时的连续监测EEG、肌电图（EMG）和活动进行了标记。在这里，我们评估了各种机器学习技术在将10秒的时间划分为三个离散类中的一个方面的性能：矛盾、慢波或尾波。我们的研究包括决策树、随机森林、朴素贝叶斯分类器、逻辑回归分类器和人工神经网络。这些方法的准确度在74%到96%之间。最值得注意的是，随机森林和人工神经网络分别获得了95.78%和93.31%的显著准确率。在这里，我们展示了各种机器学习分类器的潜力，它们可以根据单个EEG读数和单个EMG读数自动、准确、可靠地对警戒状态进行分类。 摘要：Sleep studies are imperative to recapitulate phenotypes associated with sleep loss and uncover mechanisms contributing to psychopathology. Most often, investigators manually classify the polysomnography into vigilance states, which is time-consuming, requires extensive training, and is prone to inter-scorer variability. While many works have successfully developed automated vigilance state classifiers based on multiple EEG channels, we aim to produce an automated and open-access classifier that can reliably predict vigilance state based on a single cortical electroencephalogram (EEG) from rodents to minimize the disadvantages that accompany tethering small animals via wires to computer programs. Approximately 427 hours of continuously monitored EEG, electromyogram (EMG), and activity were labeled by a domain expert out of 571 hours of total data. Here we evaluate the performance of various machine learning techniques on classifying 10-second epochs into one of three discrete classes: paradoxical, slow-wave, or wake. Our investigations include Decision Trees, Random Forests, Naive Bayes Classifiers, Logistic Regression Classifiers, and Artificial Neural Networks. These methodologies have achieved accuracies ranging from approximately 74% to approximately 96%. Most notably, the Random Forest and the ANN achieved remarkable accuracies of 95.78% and 93.31%, respectively. Here we have shown the potential of various machine learning classifiers to automatically, accurately, and reliably classify vigilance states based on a single EEG reading and a single EMG reading.
表征(2篇)
【1】 Hard Negative Sampling via Regularized Optimal Transport for Contrastive Representation Learning 标题：对比表征学习的正则化最优传输硬负采样 链接：https://arxiv.org/abs/2111.03169
作者：Ruijie Jiang,Prakash Ishwar,Shuchin Aeron 摘要：我们研究了非监督对比表征学习中硬负抽样分布的设计问题。我们分析了一个新的最小-最大框架，该框架寻求在所有耦合（受边际约束的正样本和负样本之间的联合分布）上最小化最大（最坏情况）广义对比学习损失的表示，并证明得到的最小-最大最佳表示将退化。这为在耦合上加入额外的正则化约束提供了第一个理论依据。我们通过最优输运理论的透镜重新解释最小-最大问题，并利用正则化输运耦合来控制反例的硬度。我们证明了最近提出的最新硬负采样分布是对应于耦合熵正则化的特例。 摘要：We study the problem of designing hard negative sampling distributions for unsupervised contrastive representation learning. We analyze a novel min-max framework that seeks a representation which minimizes the maximum (worst-case) generalized contrastive learning loss over all couplings (joint distributions between positive and negative samples subject to marginal constraints) and prove that the resulting min-max optimum representation will be degenerate. This provides the first theoretical justification for incorporating additional regularization constraints on the couplings. We re-interpret the min-max problem through the lens of Optimal Transport theory and utilize regularized transport couplings to control the degree of hardness of negative examples. We demonstrate that the state-of-the-art hard negative sampling distributions that were recently proposed are a special case corresponding to entropic regularization of the coupling.
【2】 Recurrent Neural Networks for Learning Long-term Temporal Dependencies with Reanalysis of Time Scale Representation 标题：基于时间尺度表示重分析的递归神经网络学习长期时间依赖性 链接：https://arxiv.org/abs/2111.03282
作者：Kentaro Ohno,Atsutoshi Kumagai 机构：NTT Computer and Data Science Laboratories 备注：8 pages, 5 figures, IEEE ICBK 2021 摘要：具有选通机制（如LSTM或GRU）的递归神经网络是建模序列数据的强大工具。在该机制中，为了控制RNN中处于隐藏状态的信息流而引入的遗忘门最近被重新解释为状态的时间尺度的代表，即RNN保留输入信息的时间长度的度量。在此解释的基础上，提出了几种参数初始化方法来利用数据中时间依赖性的先验知识，以提高可学习性。然而，解释依赖于各种不切实际的假设，例如在某个时间点之后没有输入。在这部作品中，我们在一个更现实的环境中重新考虑对遗忘之门的解释。首先，我们总结现有的门控RNN理论，以便我们可以考虑的情况下，连续输入。然后，我们认为，当相对于状态的损失梯度随着时间的推移呈指数下降时，遗忘门作为时间表示的解释是有效的。我们的经验证明，现有的RNN在几个任务的初始训练阶段满足这个梯度条件，这与以前的初始化方法是一致的。基于这一发现，我们提出了一种构建新的RNN的方法，它可以比传统模型代表更长的时间尺度，这将提高长期序列数据的可学习性。通过对真实数据集的实验验证了该方法的有效性。 摘要：Recurrent neural networks with a gating mechanism such as an LSTM or GRU are powerful tools to model sequential data. In the mechanism, a forget gate, which was introduced to control information flow in a hidden state in the RNN, has recently been re-interpreted as a representative of the time scale of the state, i.e., a measure how long the RNN retains information on inputs. On the basis of this interpretation, several parameter initialization methods to exploit prior knowledge on temporal dependencies in data have been proposed to improve learnability. However, the interpretation relies on various unrealistic assumptions, such as that there are no inputs after a certain time point. In this work, we reconsider this interpretation of the forget gate in a more realistic setting. We first generalize the existing theory on gated RNNs so that we can consider the case where inputs are successively given. We then argue that the interpretation of a forget gate as a temporal representation is valid when the gradient of loss with respect to the state decreases exponentially as time goes back. We empirically demonstrate that existing RNNs satisfy this gradient condition at the initial training phase on several tasks, which is in good agreement with previous initialization methods. On the basis of this finding, we propose an approach to construct new RNNs that can represent a longer time scale than conventional models, which will improve the learnability for long-term sequential data. We verify the effectiveness of our method by experiments with real-world datasets.
优化|敛散性(4篇)
【1】 Risk-averse Heteroscedastic Bayesian Optimization 标题：风险厌恶的异方差贝叶斯优化 链接：https://arxiv.org/abs/2111.03637
作者：Anastasiia Makarova,Ilnura Usmanova,Ilija Bogunovic,Andreas Krause 机构：ETH Zürich 摘要：许多高风险应用中出现的黑盒优化任务需要风险规避决策。然而，标准的贝叶斯优化（BO）范式只优化期望值。我们将BO推广到目标的交易均值和与输入相关的方差，我们假设这两个变量都是先验未知的。特别是，我们提出了一种新的风险规避异方差贝叶斯优化算法（RAHBO），该算法旨在识别具有高回报和低噪声方差的解，同时动态学习噪声分布。为此，我们将期望和方差都建模为（未知）RKHS函数，并提出了一种新的风险感知获取函数。我们对我们的方法感到遗憾，并提供了一个健壮的规则，用于报告只需确定单个解决方案的应用程序的最终决策点。我们证明了RAHBO在综合基准函数和超参数调整任务上的有效性。 摘要：Many black-box optimization tasks arising in high-stakes applications require risk-averse decisions. The standard Bayesian optimization (BO) paradigm, however, optimizes the expected value only. We generalize BO to trade mean and input-dependent variance of the objective, both of which we assume to be unknown a priori. In particular, we propose a novel risk-averse heteroscedastic Bayesian optimization algorithm (RAHBO) that aims to identify a solution with high return and low noise variance, while learning the noise distribution on the fly. To this end, we model both expectation and variance as (unknown) RKHS functions, and propose a novel risk-aware acquisition function. We bound the regret for our approach and provide a robust rule to report the final decision point for applications where only a single solution must be identified. We demonstrate the effectiveness of RAHBO on synthetic benchmark functions and hyperparameter tuning tasks.
【2】 Contextual Bayesian optimization with binary outputs 标题：具有二进制输出的上下文贝叶斯优化 链接：https://arxiv.org/abs/2111.03447
作者：Tristan Fauvel,Matthew Chalk 机构：Sorbonne Universit´e, INSERM, CNRS, Institut de la Vision, F-, Paris, France 摘要：贝叶斯优化（BO）是一种优化昂贵的黑盒函数的有效方法。它已被推广到目标函数评估返回随机二元反馈的场景，如给定测试中的成功/失败，或不同参数设置之间的偏好。在许多实际情况下，可以在直接影响观测的受控“环境”或“环境”中评估目标函数。例如，可以直接改变用于评估系统性能的测试的“难度”。通过二元反馈，上下文确定从每次观察中获得的信息。例如，如果测试太容易/太难，系统总是会成功/失败，产生非信息性的二进制输出。在这里，我们结合贝叶斯主动学习和优化的思想，在每次迭代中有效地选择最佳上下文和优化参数。我们演示了我们算法的性能，并说明了如何将其用于解决视觉心理物理学中的一个具体应用：使用心理物理学测量，通过矫正镜片有效地改善患者的视力。 摘要：Bayesian optimization (BO) is an efficient method to optimize expensive black-box functions. It has been generalized to scenarios where objective function evaluations return stochastic binary feedback, such as success/failure in a given test, or preference between different parameter settings. In many real-world situations, the objective function can be evaluated in controlled 'contexts' or 'environments' that directly influence the observations. For example, one could directly alter the 'difficulty' of the test that is used to evaluate a system's performance. With binary feedback, the context determines the information obtained from each observation. For example, if the test is too easy/hard, the system will always succeed/fail, yielding uninformative binary outputs. Here we combine ideas from Bayesian active learning and optimization to efficiently choose the best context and optimization parameter on each iteration. We demonstrate the performance of our algorithm and illustrate how it can be used to tackle a concrete application in visual psychophysics: efficiently improving patients' vision via corrective lenses, using psychophysics measurements.
【3】 Rate of Convergence of Polynomial Networks to Gaussian Processes 标题：多项式网络对高斯过程的收敛速度 链接：https://arxiv.org/abs/2111.03175
作者：Adam Klukowski 机构：Huawei Noah’s Ark Lab 备注：23 pages (13 for the main body) 摘要：我们研究了具有随机权重的单隐层神经网络。众所周知，在无限多个神经元的限制下，它们简化为高斯过程。对于具有多项式激活的网络，我们证明了2-Wasserstein度量的这种收敛速度是$O（n^{-frac{1}{2}）$，其中$n$是隐藏神经元的数量。我们怀疑这个比率是渐进的。我们改进了其他激活的已知收敛速度，ReLU的幂律为$n$，erf的平方根逆为对数因子。我们探讨了球谐函数、Stein核和非各向同性环境下最优输运之间的相互作用。 摘要：We examine one-hidden-layer neural networks with random weights. It is well-known that in the limit of infinitely many neurons they simplify to Gaussian processes. For networks with a polynomial activation, we demonstrate that the rate of this convergence in 2-Wasserstein metric is $O(n^{-frac{1}{2}})$, where $n$ is the number of hidden neurons. We suspect this rate is asymptotically sharp. We improve the known convergence rate for other activations, to power-law in $n$ for ReLU and inverse-square-root up to logarithmic factors for erf. We explore the interplay between spherical harmonics, Stein kernels and optimal transport in the non-isotropic setting.
【4】 Multi-Objective Constrained Optimization for Energy Applications via Tree Ensembles 标题：基于树集合法的能量应用多目标约束优化 链接：https://arxiv.org/abs/2111.03140
作者：Alexander Thebelt,Calvin Tsay,Robert M. Lee,Nathan Sudermann-Merx,David Walz,Tom Tranter,Ruth Misener 机构：Imperial College London, South Kensington, SW,AZ, UK., BASF SE, Ludwigshafen am Rhein, Germany., Cooperative State University Mannheim, Mannheim, Germany., nathan-georg.sudermann-merx, Electrochemical Innovation Lab, University College London 备注：36 pages, 8 figures, 5 tables 摘要：由于强烈的非线性系统行为和多个竞争目标，例如经济收益与环境影响，能源系统优化问题非常复杂。此外，大量的输入变量和不同的变量类型（如连续变量和分类变量）是现实应用中常见的挑战。在某些情况下，建议的最佳解决方案需要遵守与物理特性或安全关键操作条件相关的显式输入约束。本文提出了一种新的基于树集合的数据驱动策略，用于具有异构变量空间的黑箱问题的约束多目标优化，其基本系统动力学要么过于复杂，无法建模，要么未知。在一个由合成基准和相关能源应用组成的广泛案例研究中，我们展示了与其他最先进工具相比，所提出算法的竞争性能和采样效率，使其成为现实世界中评估预算有限的应用的一个有用的一体化解决方案。 摘要：Energy systems optimization problems are complex due to strongly non-linear system behavior and multiple competing objectives, e.g. economic gain vs. environmental impact. Moreover, a large number of input variables and different variable types, e.g. continuous and categorical, are challenges commonly present in real-world applications. In some cases, proposed optimal solutions need to obey explicit input constraints related to physical properties or safety-critical operating conditions. This paper proposes a novel data-driven strategy using tree ensembles for constrained multi-objective optimization of black-box problems with heterogeneous variable spaces for which underlying system dynamics are either too complex to model or unknown. In an extensive case study comprised of synthetic benchmarks and relevant energy applications we demonstrate the competitive performance and sampling efficiency of the proposed algorithm compared to other state-of-the-art tools, making it a useful all-in-one solution for real-world applications with limited evaluation budgets.
预测|估计(5篇)
【1】 A Variational U-Net for Weather Forecasting 标题：一种用于天气预报的变分U网 链接：https://arxiv.org/abs/2111.03476
作者：Pak Hay Kwok,Qi Qi 机构： Weather 4cast[ 1] by the Institute of Advanced Research in ArtificialIntelligence is an open competition that challenges itsparticipants to develop algorithms to predict the futurestates of the atmosphere over specific regions 备注：6 pages, 3 figures. To be published in the proceedings of the 1st workshop on Complex Data Challenges in Earth Observation (CDCEO) 2021 摘要：从大气数据中发现模式和见解不仅可以实现更准确的天气预测，还可以提供有价值的信息帮助应对气候变化。Weather4cast是一项公开竞赛，旨在评估机器学习算法预测未来大气状态的能力。这里，我们将介绍Weather4cast的第三位解决方案。我们提出了一种新的变分U-NET，结合变分自动编码器的能力，考虑数据的概率性质与U-NET的能力恢复细粒度的细节。此解决方案从排名第四的解决方案演变为Traffic4cast 2020，具有许多共性，表明其适用于极为不同的领域，如天气和交通。 摘要：Not only can discovering patterns and insights from atmospheric data enable more accurate weather predictions, but it may also provide valuable information to help tackle climate change. Weather4cast is an open competition that aims to evaluate machine learning algorithms' capability to predict future atmospheric states. Here, we describe our third-place solution to Weather4cast. We present a novel Variational U-Net that combines a Variational Autoencoder's ability to consider the probabilistic nature of data with a U-Net's ability to recover fine-grained details. This solution is an evolution from our fourth-place solution to Traffic4cast 2020 with many commonalities, suggesting its applicability to vastly different domains, such as weather and traffic.
【2】 Transferable Time-Series Forecasting under Causal Conditional Shift 标题：因果条件转移下的可转移时间序列预测 链接：https://arxiv.org/abs/2111.03422
作者：Zijian Li,Ruichu Cai,Tom Z. J Fu,Kun Zhang 机构： Fu is with the School of Computing, Guangdong University ofTechnology 摘要：本文主要研究时间序列预测的textcolor{black}{semi-Supervisive}域自适应问题，这是一个容易被忽略但具有挑战性的问题，因为它具有多变和复杂的条件依赖性。事实上，这些特定于域的条件依赖主要由数据偏移、时间滞后和变量数据分布导致。为了解决这一问题，我们分析了时间序列数据的变分条件依赖性，并考虑因果结构在不同域之间是稳定的，并进一步提高因果条件转移假设。在这一假设的启发下，我们考虑了时间序列数据的因果生成过程，并设计了一个端到端的可传递时间序列预测模型。该方法不仅可以发现跨域的Granger因果关系，而且可以解决跨域时间序列预测问题。它甚至可以在一定程度上提供预测结果的可解释性。我们进一步从理论上分析了所提出方法的优越性，其中目标域上的泛化误差不仅受到源域和目标域上的经验风险的限制，而且还受到来自不同域的因果结构之间的相似性的限制。在合成数据和真实数据上的实验结果表明了该方法对可转移时间序列预测的有效性。 摘要：This paper focuses on the problem of textcolor{black}{semi-supervised} domain adaptation for time-series forecasting, which is an easily neglected but challenging problem due to the changeable and complex conditional dependencies. In fact, these domain-specific conditional dependencies are mainly led by the data offset, the time lags, and the variant data distribution. In order to cope with this problem, we analyze the variational conditional dependencies in time-series data and consider that the causal structures are stable among different domains, and further raise the causal conditional shift assumption. Enlightened by this assumption, we consider the causal generation process for time-series data and devise an end-to-end model for transferable time-series forecasting. The proposed method can not only discover the cross-domain textit{Granger Causality} but also address the cross-domain time-series forecasting problem. It can even provide the interpretability of the predicted results to some extent. We further theoretically analyze the superiority of the proposed methods, where the generalization error on the target domain is not only bounded by the empirical risks on the source and target domains but also by the similarity between the causal structures from different domains. Experimental results on both synthetic and real data demonstrate the effectiveness of the proposed method for transferable time-series forecasting.
【3】 Long Range Probabilistic Forecasting in Time-Series using High Order Statistics 标题：基于高阶统计量的时间序列长期概率预测 链接：https://arxiv.org/abs/2111.03394
作者：Prathamesh Deshpande,Sunita Sarawagi 机构：IIT Bombay 备注：9 pages, 2 figures, 3 tables, 1 algorithm 摘要：长期预测是许多决策支持系统的起点，这些系统需要从预测值的高级聚合模式中进行推理。最先进的时间序列预测方法要么在长期预测中受到概念漂移的影响，要么无法准确预测连贯和准确的高层总量。在这项工作中，我们提出了一种新的概率预测方法，该方法产生的预测在基本水平和预测的聚合统计方面是一致的。我们使用一种新的推理方法实现了预测基准水平和聚合统计数据之间的一致性。我们的推理方法基于KL散度，可以有效地以闭合形式求解。我们表明，我们的方法提高了在三个不同领域的真实数据集上进行推理后的基础级和不可见聚合的预测性能。 摘要：Long range forecasts are the starting point of many decision support systems that need to draw inference from high-level aggregate patterns on forecasted values. State of the art time-series forecasting methods are either subject to concept drift on long-horizon forecasts, or fail to accurately predict coherent and accurate high-level aggregates. In this work, we present a novel probabilistic forecasting method that produces forecasts that are coherent in terms of base level and predicted aggregate statistics. We achieve the coherency between predicted base-level and aggregate statistics using a novel inference method. Our inference method is based on KL-divergence and can be solved efficiently in closed form. We show that our method improves forecast performance across both base level and unseen aggregates post inference on real datasets ranging three diverse domains.
【4】 EpilNet: A Novel Approach to IoT based Epileptic Seizure Prediction and Diagnosis System using Artificial Intelligence 标题：EpilNet：一种基于物联网的人工智能癫痫发作预测与诊断系统 链接：https://arxiv.org/abs/2111.03265
作者：Shivam Gupta,Virender Ranga,Priyansh Agrawal 机构：Department of Computer Science and Engineering, Indian Institute of Information Technology, Sonepat, Haryana, India (Mentor, National Institute of Technology, Kurukshetra, Haryana, India) 备注：12 Pages, 12 Figures, 2 Tables 摘要：癫痫是最常见的神经系统疾病之一。这种疾病的主要特征是频繁发作，即大脑中的电失衡。通常伴随着身体部位的晃动，甚至出现导联（晕厥）。在过去几年中，出现了许多治疗方法。这些措施主要涉及使用抗癫痫药物控制癫痫发作。但在70%的病例中，这些药物无效，当病情恶化时，手术是唯一的解决办法。因此，患者在癫痫发作时需要照顾好自己，确保安全。可穿戴脑电图（EEG）设备是随着医学科学和技术的发展而出现的。这些设备有助于分析大脑的电活动。脑电图有助于定位受影响的皮层区域。最重要的是，它可以提前在现场预测任何癫痫发作。这导致对有效和高效的癫痫预测和诊断系统的需求突然增加。本文提出了一种新的癫痫发作预测和诊断系统EpilNet。它是一个一维（1D）卷积神经网络。EpilNet对五个等级的测试准确率为79.13%，与相关工作相比，显著提高了约6-7%。开发的Web API有助于EpilNet的实际使用。因此，它是一个病人和医生的综合系统。该系统将帮助患者预防受伤或事故，并提高医院医生治疗过程的效率。 摘要：Epilepsy is one of the most occurring neurological diseases. The main characteristic of this disease is a frequent seizure, which is an electrical imbalance in the brain. It is generally accompanied by shaking of body parts and even leads (fainting). In the past few years, many treatments have come up. These mainly involve the use of anti-seizure drugs for controlling seizures. But in 70% of cases, these drugs are not effective, and surgery is the only solution when the condition worsens. So patients need to take care of themselves while having a seizure and be safe. Wearable electroencephalogram (EEG) devices have come up with the development in medical science and technology. These devices help in the analysis of brain electrical activities. EEG helps in locating the affected cortical region. The most important is that it can predict any seizure in advance on-site. This has resulted in a sudden increase in demand for effective and efficient seizure prediction and diagnosis systems. A novel approach to epileptic seizure prediction and diagnosis system EpilNet is proposed in the present paper. It is a one-dimensional (1D) convolution neural network. EpilNet gives the testing accuracy of 79.13% for five classes, leading to a significant increase of about 6-7% compared to related works. The developed Web API helps in bringing EpilNet into practical use. Thus, it is an integrated system for both patients and doctors. The system will help patients prevent injury or accidents and increase the efficiency of the treatment process by doctors in the hospitals.
【5】 Predictive Machine Learning of Objective Boundaries for Solving COPs 标题：求解复杂系统问题的目标边界预测机器学习 链接：https://arxiv.org/abs/2111.03160
作者：Helge Spieker,Arnaud Gotlieb 备注：None 摘要：通过边界估计，即提供成本函数的严格边界，可以大大简化约束优化问题（COP）的求解。通过向有监督机器学习（ML）模型提供由已知边界和COP提取特征组成的数据，可以训练该模型来估计新COP实例的边界。在本文中，我们首先概述了约束编程（CP）的ML的现有知识体系，它从问题实例中学习。其次，我们介绍了一个边界估计框架，该框架被用作支持CP解算器的工具。在此框架内，讨论和评估了不同的ML模型对边界估计的适用性，并给出了避免不可行估计的对策，以避免解算器找到最优解。第三，我们在七个COP上进行了不同CP解算器的实验研究。我们的结果表明，对于这些COP，可以用很少的开销学习接近最优的边界。这些估计的边界将目标域大小减少60-88%，并可帮助解算器在搜索过程中尽早找到接近最优的解。 摘要：Solving Constraint Optimization Problems (COPs) can be dramatically simplified by boundary estimation, that is, providing tight boundaries of cost functions. By feeding a supervised Machine Learning (ML) model with data composed of known boundaries and extracted features of COPs, it is possible to train the model to estimate boundaries of a new COP instance. In this paper, we first give an overview of the existing body of knowledge on ML for Constraint Programming (CP) which learns from problem instances. Second, we introduce a boundary estimation framework that is applied as a tool to support a CP solver. Within this framework, different ML models are discussed and evaluated regarding their suitability for boundary estimation, and countermeasures to avoid unfeasible estimations that avoid the solver to find an optimal solution are shown. Third, we present an experimental study with distinct CP solvers on seven COPs. Our results show that near-optimal boundaries can be learned for these COPs with only little overhead. These estimated boundaries reduce the objective domain size by 60-88% and can help the solver to find near-optimal solutions early during search.
其他神经网络|深度学习|模型|建模(12篇)
【1】 NAS-Bench-x11 and the Power of Learning Curves 标题：NAS-BENCH-X11和学习曲线的力量 链接：https://arxiv.org/abs/2111.03602
作者：Shen Yan,Colin White,Yash Savani,Frank Hutter 机构： Michigan State University, Abacus.AI, Carnegie Mellon University, University of Freiburg, Bosch Center for Artificial Intelligence 备注：NeurIPS 2021 摘要：虽然神经架构搜索（NAS）的早期研究需要极端的计算资源，但最近发布的表格和替代基准大大提高了NAS研究的速度和再现性。然而，两个最流行的基准并没有为每个体系结构提供完整的训练信息。因此，在这些基准上，不可能运行许多类型的多保真度技术，例如需要在任意时期评估体系结构的学习曲线外推。在这项工作中，我们提出了一种使用奇异值分解和噪声建模的方法，以创建替代基准NAS-Bench-111、NAS-Bench-311和NAS-Bench-NLP11，它们输出每个体系结构的完整训练信息，而不仅仅是最终的验证精度。我们通过引入一个学习曲线外推框架来修改单保真度算法，展示了使用完整训练信息的能力，表明它比流行的单保真度算法有所改进，后者在发布时声称是最先进的。我们的代码和预训练模型可在https://github.com/automl/nas-bench-x11. 摘要：While early research in neural architecture search (NAS) required extreme computational resources, the recent releases of tabular and surrogate benchmarks have greatly increased the speed and reproducibility of NAS research. However, two of the most popular benchmarks do not provide the full training information for each architecture. As a result, on these benchmarks it is not possible to run many types of multi-fidelity techniques, such as learning curve extrapolation, that require evaluating architectures at arbitrary epochs. In this work, we present a method using singular value decomposition and noise modeling to create surrogate benchmarks, NAS-Bench-111, NAS-Bench-311, and NAS-Bench-NLP11, that output the full training information for each architecture, rather than just the final validation accuracy. We demonstrate the power of using the full training information by introducing a learning curve extrapolation framework to modify single-fidelity algorithms, showing that it leads to improvements over popular single-fidelity algorithms which claimed to be state-of-the-art upon release. Our code and pretrained models are available at https://github.com/automl/nas-bench-x11.
【2】 Visualizing the Emergence of Intermediate Visual Patterns in DNNs 标题：在DNNs中可视化中间视觉模式的出现 链接：https://arxiv.org/abs/2111.03505
作者：Mingjie Li,Shaobo Wang,Quanshi Zhang 机构：Shanghai Jiao Tong University, Harbin Institute of Technology 摘要：本文提出了一种用DNN编码的中间层视觉模式识别能力的可视化方法。具体而言，我们可视化了（1）DNN如何在训练过程中逐渐学习每个中间层中的区域视觉模式，以及（2）DNN使用低层中的非辨别模式通过前向传播在中/高层中构建离散模式的效果。基于我们的可视化方法，我们可以量化DNN学习到的知识点（即辨别性视觉模式的数量），以评估DNN的表示能力。此外，该方法还为现有深度学习技术的信号处理行为提供了新的见解，如对抗性攻击和知识提取。 摘要：This paper proposes a method to visualize the discrimination power of intermediate-layer visual patterns encoded by a DNN. Specifically, we visualize (1) how the DNN gradually learns regional visual patterns in each intermediate layer during the training process, and (2) the effects of the DNN using non-discriminative patterns in low layers to construct disciminative patterns in middle/high layers through the forward propagation. Based on our visualization method, we can quantify knowledge points (i.e., the number of discriminative visual patterns) learned by the DNN to evaluate the representation capacity of the DNN. Furthermore, this method also provides new insights into signal-processing behaviors of existing deep-learning techniques, such as adversarial attacks and knowledge distillation.
【3】 Deep-Learning Based Linear Precoding for MIMO Channels with Finite-Alphabet Signaling 标题：基于深度学习的有限字母MIMO信道线性预编码 链接：https://arxiv.org/abs/2111.03504
作者：Maksym A. Girnyk 备注：None 摘要：本文研究了采用有限字母信令的多输入多输出（MIMO）通信信道的线性预编码问题。由于星座约束互信息的计算代价高昂，现有的解决方案通常具有较高的计算复杂度。与现有工作相比，本文采用了不同的方法来解决MIMO预编码问题。即，提出了一种基于深度学习的数据驱动方法。在离线训练阶段，深度神经网络在一组MIMO信道矩阵上学习最优解。这允许在在线推断阶段降低预编码器优化的计算复杂性。数值结果表明，与现有的预编码算法相比，所提出的解决方案在显著降低复杂度和接近最优性能方面是有效的。 摘要：This paper studies the problem of linear precoding for multiple-input multiple-output (MIMO) communication channels employing finite-alphabet signaling. Existing solutions typically suffer from high computational complexity due to costly computations of the constellation-constrained mutual information. In contrast to existing works, this paper takes a different path of tackling the MIMO precoding problem. Namely, a data-driven approach, based on deep learning, is proposed. In the offline training phase, a deep neural network learns the optimal solution on a set of MIMO channel matrices. This allows the reduction of the computational complexity of the precoder optimization in the online inference phase. Numerical results demonstrate the efficiency of the proposed solution vis-`a-vis existing precoding algorithms in terms of significantly reduced complexity and close-to-optimal performance.
【4】 Learning Large Neighborhood Search Policy for Integer Programming 标题：整数规划的大邻域搜索学习策略 链接：https://arxiv.org/abs/2111.03466
作者：Yaoxin Wu,Wen Song,Zhiguang Cao,Jie Zhang 机构：Nanyang Technological University, Singapore, Shandong University, Qingdao, China, Singapore Institute of Manufacturing Technology, ASTAR, Singapore 备注：18 pages, 4 figure 摘要：我们提出了一种深度强化学习（RL）方法来学习整数规划（IP）的大邻域搜索（LNS）策略。RL策略被训练为销毁操作符，以在每个步骤中选择变量子集，该子集由IP解算器作为修复操作符重新优化。然而，可变子集的组合数量阻碍了典型RL算法的直接应用。为了应对这一挑战，我们通过将所有子集分解为每个变量的二进制决策来表示它们。然后，我们设计了一个神经网络来并行学习每个变量的策略，并通过定制的actor-critic算法进行训练。我们在四个具有代表性的IP问题上对所提出的方法进行了评估。结果表明，它可以在更短的时间内找到比SCIP更好的解决方案，并且显著优于具有相同运行时间的其他LNS基线。此外，当这些政策推广到更大的问题时，这些优势明显存在。使用Gurobi进行的进一步实验还表明，我们的方法可以在相同的时间限制内优于这种最先进的商业求解器。 摘要：We propose a deep reinforcement learning (RL) method to learn large neighborhood search (LNS) policy for integer programming (IP). The RL policy is trained as the destroy operator to select a subset of variables at each step, which is reoptimized by an IP solver as the repair operator. However, the combinatorial number of variable subsets prevents direct application of typical RL algorithms. To tackle this challenge, we represent all subsets by factorizing them into binary decisions on each variable. We then design a neural network to learn policies for each variable in parallel, trained by a customized actor-critic algorithm. We evaluate the proposed method on four representative IP problems. Results show that it can find better solutions than SCIP in much less time, and significantly outperform other LNS baselines with the same runtime. Moreover, these advantages notably persist when the policies generalize to larger problems. Further experiments with Gurobi also reveal that our method can outperform this state-of-the-art commercial solver within the same time limit.
【5】 Online Learning in Periodic Zero-Sum Games 标题：周期零和博弈中的在线学习 链接：https://arxiv.org/abs/2111.03377
作者：Tanner Fiez,Ryann Sim,Stratis Skoulakis,Georgios Piliouras,Lillian Ratliff 机构：University of Washington, Seattle, Washington, SUTD, Singapore 备注：To appear at NeurIPS 2021 摘要：博弈论的一个重要成果是冯·诺依曼的极小极大定理，该定理指出零和博弈允许一个本质上唯一的均衡解。经典的学习结果建立在这个定理的基础上，证明了零和博弈中在线无悔动态在时间平均意义下收敛到一个均衡。在过去几年中，一个关键的研究方向集中在描述这种动态的日常行为。这方面的一般结果表明，在零和博弈中，广泛的在线学习动态是循环的，形式上是Poincar{e}循环的。我们分析了这些在线学习行为在具有时不变平衡点的周期零和博弈情况下的鲁棒性。该模型概括了通常的重复博弈公式，同时也是玩家之间重复竞争的现实和自然模型，这种竞争依赖于外部环境变化，如时间效应、每周趋势和季节性。有趣的是，即使在最简单的情况下，时间平均收敛也可能失败，尽管平衡点是固定的。相比之下，使用新的分析方法，我们证明了Poincar'{e}递归可证明推广，尽管这些动力系统的复杂性、非自治性。 摘要：A seminal result in game theory is von Neumann's minmax theorem, which states that zero-sum games admit an essentially unique equilibrium solution. Classical learning results build on this theorem to show that online no-regret dynamics converge to an equilibrium in a time-average sense in zero-sum games. In the past several years, a key research direction has focused on characterizing the day-to-day behavior of such dynamics. General results in this direction show that broad classes of online learning dynamics are cyclic, and formally Poincar'{e} recurrent, in zero-sum games. We analyze the robustness of these online learning behaviors in the case of periodic zero-sum games with a time-invariant equilibrium. This model generalizes the usual repeated game formulation while also being a realistic and natural model of a repeated competition between players that depends on exogenous environmental variations such as time-of-day effects, week-to-week trends, and seasonality. Interestingly, time-average convergence may fail even in the simplest such settings, in spite of the equilibrium being fixed. In contrast, using novel analysis methods, we show that Poincar'{e} recurrence provably generalizes despite the complex, non-autonomous nature of these dynamical systems.
【6】 Confidential Machine Learning Computation in Untrusted Environments: A Systems Security Perspective 标题：不可信环境中的机密性机器学习计算：系统安全角度 链接：https://arxiv.org/abs/2111.03308
作者：Kha Dinh Duy,Taehyun Noh,Siwon Huh,Hojoon Lee 摘要：随着机器学习（ML）技术和应用正在迅速改变许多计算领域，与ML相关的安全问题也在出现。在系统安全领域，人们已经做出了许多努力来确保ML模型和数据的机密性。ML计算通常不可避免地在不受信任的环境中执行，并且需要复杂的多方安全需求。因此，研究人员利用可信执行环境（TEE）来构建机密的ML计算系统。本文通过对不可信环境下受TEE保护的机密ML计算中的攻击向量分类和缓解措施进行了系统全面的调查，分析了多方ML安全需求，并讨论了相关的工程挑战。 摘要：As machine learning (ML) technologies and applications are rapidly changing many domains of computing, security issues associated with ML are also emerging. In the domain of systems security, many endeavors have been made to ensure ML model and data confidentiality. ML computations are often inevitably performed in untrusted environments and entail complex multi-party security requirements. Hence, researchers have leveraged the Trusted Execution Environments (TEEs) to build confidential ML computation systems. This paper conducts a systematic and comprehensive survey by classifying attack vectors and mitigation in TEE-protected confidential ML computation in the untrusted environment, analyzes the multi-party ML security requirements, and discusses related engineering challenges.
【7】 Dynamic Data Augmentation with Gating Networks 标题：利用门控网络实现动态数据增强 链接：https://arxiv.org/abs/2111.03253
作者：Daisuke Oba,Shinnosuke Matsuo,Brian Kenji Iwana 机构：Department of Advanced Information Technology, Kyushu University, Fukuoka, Japan 备注：submitted to ICASSP2022 摘要：数据扩充是一种通过增加数据集大小来提高机器学习方法泛化能力的技术。但是，由于每种增强方法对每个数据集都不是同等有效的，因此需要仔细选择最佳方法。我们提出了一种神经网络，该网络使用互利的选通网络和特征一致性损失动态选择最佳组合。门控网络能够控制每个数据扩充中有多少用于网络内的表示。另一方面，特征一致性损失提供了一个约束条件，即来自相同输入的增强特征应处于相似状态。在实验中，我们在2018年UCR时间序列存档的12个最大的时间序列数据集上证明了该方法的有效性，并通过对该方法的分析揭示了数据扩充方法之间的关系。 摘要：Data augmentation is a technique to improve the generalization ability of machine learning methods by increasing the size of the dataset. However, since every augmentation method is not equally effective for every dataset, you need to carefully select the best method. We propose a neural network that dynamically selects the best combination using a mutually beneficial gating network and a feature consistency loss. The gating network is able to control how much of each data augmentation is used for the representation within the network. The feature consistency loss, on the other hand, gives a constraint that augmented features from the same input should be in similar. In experiments, we demonstrate the effectiveness of the proposed method on the 12 largest time-series datasets from 2018 UCR Time Series Archive and reveal the relationships between the data augmentation methods through analysis of the proposed method.
【8】 Artificial Neural Network-Based Voltage Control of DC/DC Converter for DC Microgrid Applications 标题：基于人工神经网络的直流微电网DC/DC变换器电压控制 链接：https://arxiv.org/abs/2111.03207
作者：Hussain Sarwar Khan,Ihab S. Mohamed,Kimmo Kauhaniemi,Lantao Liu 机构：∗School of Technology and Innovations, University of Vaasa, Vaasa, Finland, †Luddy School of Informatics, Computing, and Engineering, Indiana University, Bloomington, IN , USA 备注：This paper has been accepted for publication at the 6th IEEE Workshop on the Electronic Grid (eGrid 2021). It has 6 pages, 9 figures, 2 tables 摘要：可再生能源技术的迅速发展使得微电网的概念在电力系统中得到广泛接受。由于直流配电系统具有易于集成储能、系统损耗小等优点，因此直流MG受到了广泛的关注。PI、PID等线性控制器在电力电子行业已经成熟并得到广泛应用，但随着系统参数的变化，它们的性能并不是最优的。本文提出了一种基于人工神经网络（ANN）的DC-DC升压变换器电压控制策略。在本文中，模型预测控制（MPC）被用作专家，为训练所提出的人工神经网络提供数据。当人工神经网络被微调时，它直接用于控制升压直流变换器。人工神经网络的主要优点是，即使参数不准确，神经网络系统辨识也能降低系统模型的不精确性，并且由于其并行结构，与MPC相比，它的计算量更小。为了验证所提出的人工神经网络的性能，进行了大量的MATLAB/Simulink仿真。仿真结果表明，与PI控制器相比，基于神经网络的控制策略在不同负载条件下具有更好的性能。训练后的神经网络模型的精度约为97%，适合于直流微电网的应用。 摘要：The rapid growth of renewable energy technology enables the concept of microgrid (MG) to be widely accepted in the power systems. Due to the advantages of the DC distribution system such as easy integration of energy storage and less system loss, DC MG attracts significant attention nowadays. The linear controller such as PI or PID is matured and extensively used by the power electronics industry, but their performance is not optimal as system parameters are changed. In this study, an artificial neural network (ANN) based voltage control strategy is proposed for the DC-DC boost converter. In this paper, the model predictive control (MPC) is used as an expert, which provides the data to train the proposed ANN. As ANN is tuned finely, then it is utilized directly to control the step-up DC converter. The main advantage of the ANN is that the neural network system identification decreases the inaccuracy of the system model even with inaccurate parameters and has less computational burden compared to MPC due to its parallel structure. To validate the performance of the proposed ANN, extensive MATLAB/Simulink simulations are carried out. The simulation results show that the ANN-based control strategy has better performance under different loading conditions comparison to the PI controller. The accuracy of the trained ANN model is about 97%, which makes it suitable to be used for DC microgrid applications.
【9】 MIRACLE: Causally-Aware Imputation via Learning Missing Data Mechanisms 标题：奇迹：通过学习丢失数据机制的因果意识归罪 链接：https://arxiv.org/abs/2111.03187
作者：Trent Kyono,Yao Zhang,Alexis Bellot,Mihaela van der Schaar 机构：University of California, Los Angeles, University of Cambridge, University of Oxford, Oxford, United Kingdom, The Alan Turing Institute 摘要：数据缺失是机器学习实践中的一个重要问题。从插补方法应保持数据因果结构的前提出发，我们开发了一个正则化方案，鼓励任何基线插补方法与基础数据生成机制因果一致。我们的建议是一个因果意识插补算法（MIRACLE）。MIRACLE通过同时建模缺失生成机制，反复完善基线插补，鼓励插补与数据的因果结构一致。我们在合成数据集和各种公开数据集上进行了广泛的实验，以表明MIRACLE能够在所有三种缺失情况下持续改进各种基准方法的插补：随机、完全随机和非随机。 摘要：Missing data is an important problem in machine learning practice. Starting from the premise that imputation methods should preserve the causal structure of the data, we develop a regularization scheme that encourages any baseline imputation method to be causally consistent with the underlying data generating mechanism. Our proposal is a causally-aware imputation algorithm (MIRACLE). MIRACLE iteratively refines the imputation of a baseline by simultaneously modeling the missingness generating mechanism, encouraging imputation to be consistent with the causal structure of the data. We conduct extensive experiments on synthetic and a variety of publicly available datasets to show that MIRACLE is able to consistently improve imputation over a variety of benchmark methods across all three missingness scenarios: at random, completely at random, and not at random.
【10】 Infinite Time Horizon Safety of Bayesian Neural Networks 标题：贝叶斯神经网络的无限时域安全性 链接：https://arxiv.org/abs/2111.03165
作者：Mathias Lechner,Đorđe Žikelić,Krishnendu Chatterjee,Thomas A. Henzinger 机构：IST Austria, Klosterneuburg, Austria, Ðor ¯de Žikeli´c∗ 备注：To appear in NeurIPS 2021 摘要：贝叶斯神经网络（BNN）将分布置于神经网络的权重之上，以模拟数据和网络预测中的不确定性。我们考虑在无限时域系统的反馈回路中运行贝叶斯神经网络策略时验证安全性的问题。与现有的基于采样的方法（不适用于无限时域设置）相比，我们训练了一个独立的确定性神经网络，作为无限时域安全证书。特别地，我们证明了证书网络在BNN支持的子集上保证了系统的安全性。我们的方法首先计算一个安全权重集，然后改变BNN的后权重以拒绝该集之外的样本。此外，我们还展示了如何将我们的方法扩展到安全的探索强化学习环境，以避免在政策训练期间出现不安全的轨迹。我们在一系列强化学习基准上评估我们的方法，包括非李雅普诺夫安全规范。 摘要：Bayesian neural networks (BNNs) place distributions over the weights of a neural network to model uncertainty in the data and the network's prediction. We consider the problem of verifying safety when running a Bayesian neural network policy in a feedback loop with infinite time horizon systems. Compared to the existing sampling-based approaches, which are inapplicable to the infinite time horizon setting, we train a separate deterministic neural network that serves as an infinite time horizon safety certificate. In particular, we show that the certificate network guarantees the safety of the system over a subset of the BNN weight posterior's support. Our method first computes a safe weight set and then alters the BNN's weight posterior to reject samples outside this set. Moreover, we show how to extend our approach to a safe-exploration reinforcement learning setting, in order to avoid unsafe trajectories during the training of the policy. We evaluate our approach on a series of reinforcement learning benchmarks, including non-Lyapunovian safety specifications.
【11】 Secure Machine Learning in the Cloud Using One Way Scrambling by Deconvolution 标题：利用单向解卷积置乱实现云中机器学习的安全 链接：https://arxiv.org/abs/2111.03125
作者：Yiftach Savransky,Roni Mateless,Gilad Katz 机构：Ben-Gurion University of the Negev 摘要：基于云的机器学习服务（CMLS）使企业能够利用针对大量数据预先训练的高级模型。然而，使用这些服务的主要缺点是难以保持传输数据的私密性和安全性。非对称加密要求数据在云中解密，而同态加密通常速度太慢，难以实现。我们提出了一种基于反褶积的单向置乱（OWSD）框架，它以计算开销的一小部分提供了同态加密的优点。对多个图像数据集的广泛评估表明，当CMLS的输出向量足够大时，OWSD能够实现近乎完美的分类性能。此外，我们还对我们的方法的稳健性进行了实证分析。 摘要：Cloud-based machine learning services (CMLS) enable organizations to take advantage of advanced models that are pre-trained on large quantities of data. The main shortcoming of using these services, however, is the difficulty of keeping the transmitted data private and secure. Asymmetric encryption requires the data to be decrypted in the cloud, while Homomorphic encryption is often too slow and difficult to implement. We propose One Way Scrambling by Deconvolution (OWSD), a deconvolution-based scrambling framework that offers the advantages of Homomorphic encryption at a fraction of the computational overhead. Extensive evaluation on multiple image datasets demonstrates OWSD's ability to achieve near-perfect classification performance when the output vector of the CMLS is sufficiently large. Additionally, we provide empirical analysis of the robustness of our approach.
【12】 Machine Learning Product State Distributions from Initial Reactant States for a Reactive Atom-Diatom Collision System 标题：反应性原子-硅原子碰撞体系初始反应物状态的机器学习产物态分布 链接：https://arxiv.org/abs/2111.03563
作者：Julian Arnold,Juan Carlos San Vicente Veliz,Debasish Koner,Narendra Singh,Raymond J. Bemish,Markus Meuwly 机构：)Department of Physics, University of Basel, Klingelbergstrasse , CH-, Basel, )Department of Chemistry, University of Basel, Klingelbergstrasse , CH-, Basel, Switzerland, )Department of Chemistry, Indian Institute of Science Education and 摘要：本文提出了一个机器学习（ML）模型，用于预测反应性原子-硅藻碰撞中从特定初始状态（状态到分布或STD）的产物状态分布，并对N（$^4$S） O${2}$（X$^3Sigma{rm g}^{-}$）rightarrow$NO（X$^2Pi$） O（$^3$P）反应进行了定量测试。用于训练神经网络（NN）的参考数据集包括根据$sim 2000$初始条件的显式准经典轨迹（QCT）模拟确定的最终状态分布。总的来说，通过均方根差$（sim 0.003）$和$R^2$（sim 0.99）量化的预测精度对于试验装置和离网状态特定初始条件，以及根据以平动、旋转和振动温度为特征的反应物状态分布得出的初始条件，参考QCT和STD模型预测之间的$较高。与在相同初始状态分布上评估的更粗粒度分布到分布（DTD）模型相比，STD模型显示出与反应物制备中状态分辨率的额外优势相当的性能。从特定的初始状态开始，最终状态分布的范围也更加多样化，这需要使用比DTD更具表现力的神经网络。显式QCT模拟、STD模型和广泛使用的Larsen Borgnakke（LB）模型之间的直接比较表明，STD模型是定量的，而LB模型对于旋转分布$P（j'）$最多是定性的，对于振动分布$P（v'）$则失败。因此，STD模型非常适合于模拟非平衡高速流，例如，使用直接模拟蒙特卡罗方法。 摘要：A machine learned (ML) model for predicting product state distributions from specific initial states (state-to-distribution or STD) for reactive atom-diatom collisions is presented and quantitatively tested for the N($^4$S) O$_{2}$(X$^3 Sigma_{rm g}^{-}$) $rightarrow$ NO(X$^2Pi$) O($^3$P) reaction. The reference data set for training the neural network (NN) consists of final state distributions determined from explicit quasi-classical trajectory (QCT) simulations for $sim 2000$ initial conditions. Overall, the prediction accuracy as quantified by the root-mean-squared difference $(sim 0.003)$ and the $R^2$ $(sim 0.99)$ between the reference QCT and predictions of the STD model is high for the test set and off-grid state specific initial conditions and for initial conditions drawn from reactant state distributions characterized by translational, rotational and vibrational temperatures. Compared with a more coarse grained distribution-to-distribution (DTD) model evaluated on the same initial state distributions, the STD model shows comparable performance with the additional benefit of the state resolution in the reactant preparation. Starting from specific initial states also leads to a more diverse range of final state distributions which requires a more expressive neural network to be used compared with DTD. Direct comparison between explicit QCT simulations, the STD model, and the widely used Larsen-Borgnakke (LB) model shows that the STD model is quantitative whereas the LB model is qualitative at best for rotational distributions $P(j')$ and fails for vibrational distributions $P(v')$. As such the STD model can be well-suited for simulating nonequilibrium high-speed flows, e.g., using the direct simulation Monte Carlo method.
其他(16篇)
【1】 Increasing Fairness in Predictions Using Bias Parity Score Based Loss Function Regularization 标题：使用基于偏差奇偶校验分数的损失函数正则化提高预测的公平性 链接：https://arxiv.org/abs/2111.03638
作者：Bhanu Jain,Manfred Huber,Ramez Elmasri 摘要：越来越多地利用基于机器学习的决策支持系统强调了对所有利益相关者都准确和公平的预测结果的需要。在这项工作中，我们提出了一种新的方法，以提高神经网络模型的公平性在训练期间。我们介绍了一系列增强公平性的正则化组件，这些组件与传统的基于二进制交叉熵的精度损失一起使用。这些损失函数基于偏差平价分数（BPS），该分数有助于用单个数字量化模型中的偏差。在当前的工作中，我们研究了这些正则化组件对偏差的行为和影响。我们将其部署在累犯预测任务以及基于人口普查的成人收入数据集中。结果表明，即使在不平衡的数据集中，选择合适的公平损失函数，也可以在不降低精度的情况下减少训练模型的偏差。 摘要：Increasing utilization of machine learning based decision support systems emphasizes the need for resulting predictions to be both accurate and fair to all stakeholders. In this work we present a novel approach to increase a Neural Network model's fairness during training. We introduce a family of fairness enhancing regularization components that we use in conjunction with the traditional binary-cross-entropy based accuracy loss. These loss functions are based on Bias Parity Score (BPS), a score that helps quantify bias in the models with a single number. In the current work we investigate the behavior and effect of these regularization components on bias. We deploy them in the context of a recidivism prediction task as well as on a census-based adult income dataset. The results demonstrate that with a good choice of fairness loss function we can reduce the trained model's bias without deteriorating accuracy even in unbalanced dataset.
【2】 An Empirical Study of Neural Kernel Bandits 标题：神经核带的实证研究 链接：https://arxiv.org/abs/2111.03543
作者：Michal Lisicki,Arash Afkanpour,Graham W. Taylor 机构：University of Guelph, Vector Institute for AI, Google 备注：Presented at Workshop on Bayesian Deep Learning at NeurIPS 2021 摘要：神经强盗使从业者能够有效地处理具有非线性奖励函数的问题。虽然一般情况下，上下文盗贼通常使用高斯过程（GP）预测分布进行决策，但最成功的神经变体仅使用推导中的最后一层参数。神经核研究（NK）最近在深度网络和GPs之间建立了一种对应关系，它考虑了神经网络的所有参数，并且比大多数贝叶斯神经网络训练更有效。我们建议直接应用NK诱导分布来指导基于置信上限或汤普森抽样的策略。我们证明了NK bandits在高度非线性的结构化数据上实现了最先进的性能。此外，我们还分析了训练频率和模型划分等实际因素。我们相信，我们的工作将有助于更好地理解在应用环境中使用NK的影响。 摘要：Neural bandits have enabled practitioners to operate efficiently on problems with non-linear reward functions. While in general contextual bandits commonly utilize Gaussian process (GP) predictive distributions for decision making, the most successful neural variants use only the last layer parameters in the derivation. Research on neural kernels (NK) has recently established a correspondence between deep networks and GPs that take into account all the parameters of a NN and can be trained more efficiently than most Bayesian NNs. We propose to directly apply NK-induced distributions to guide an upper confidence bound or Thompson sampling-based policy. We show that NK bandits achieve state-of-the-art performance on highly non-linear structured data. Furthermore, we analyze practical considerations such as training frequency and model partitioning. We believe our work will help better understand the impact of utilizing NKs in applied settings.
【3】 A Data-driven Approach to Neural Architecture Search Initialization 标题：一种数据驱动的神经结构搜索初始化方法 链接：https://arxiv.org/abs/2111.03524
作者：Kalifou René Traoré,Andrés Camero,Xiao Xiang Zhu 机构： Technical University ofMunich, 2Remote Sensing Institute 备注：arXiv admin note: text overlap with arXiv:2108.09126 摘要：神经结构搜索（NAS）中的算法设计受到了广泛关注，其目的是提高性能和降低计算成本。尽管取得了巨大的进步，但很少有作者提出为NAS定制初始化技术。然而，文献表明，良好的初始解集有助于找到最优解。因此，在本研究中，我们提出了一种数据驱动技术来初始化基于群体的NAS算法。特别是，我们提出了一种两步方法。首先，我们对搜索空间执行校准的聚类分析，然后，我们提取质心并使用它们初始化NAS算法。我们使用三种基于群体的算法（即遗传算法、进化算法和老化进化算法）在CIFAR-10上对我们提出的方法进行随机和拉丁超立方体采样初始化的基准测试。更具体地说，我们使用NAS-Bench-101来利用NAS基准的可用性。结果表明，与随机和拉丁超立方体抽样相比，所提出的初始化技术能够实现两个搜索基线的显著长期改进，有时在各种搜索场景（各种训练预算）中也能实现。此外，我们分析了获得的解的分布，发现数据驱动初始化技术提供的总体能够检索高适应度和类似配置的局部最优（最大值）。 摘要：Algorithmic design in neural architecture search (NAS) has received a lot of attention, aiming to improve performance and reduce computational cost. Despite the great advances made, few authors have proposed to tailor initialization techniques for NAS. However, literature shows that a good initial set of solutions facilitate finding the optima. Therefore, in this study, we propose a data-driven technique to initialize a population-based NAS algorithm. Particularly, we proposed a two-step methodology. First, we perform a calibrated clustering analysis of the search space, and second, we extract the centroids and use them to initialize a NAS algorithm. We benchmark our proposed approach against random and Latin hypercube sampling initialization using three population-based algorithms, namely a genetic algorithm, evolutionary algorithm, and aging evolution, on CIFAR-10. More specifically, we use NAS-Bench-101 to leverage the availability of NAS benchmarks. The results show that compared to random and Latin hypercube sampling, the proposed initialization technique enables achieving significant long-term improvements for two of the search baselines, and sometimes in various search scenarios (various training budgets). Moreover, we analyze the distributions of solutions obtained and find that that the population provided by the data-driven initialization technique enables retrieving local optima (maxima) of high fitness and similar configurations.
【4】 Solving the Class Imbalance Problem Using a Counterfactual Method for Data Augmentation 标题：用一种反事实的数据增强方法解决班级不平衡问题 链接：https://arxiv.org/abs/2111.03516
作者：Mohammed Temraz,Mark T. Keane 机构：School of Computer Science, University College Dublin, Belfield, Dublin , Ireland, Insight Centre for Data Analytics, University College Dublin, Belfield, Dublin , Ireland, VistaMilk SFI Research Centre, University College Dublin, Belfield, Dublin , Ireland 备注：47 pages, 4 figures 摘要：从类不平衡数据集中学习对许多机器学习算法提出了挑战。根据定义，许多现实世界的领域都是阶级不平衡的，因为多数阶级自然比少数阶级拥有更多的实例（例如，真实的银行交易比欺诈性交易发生得更频繁）。人们提出了许多方法来解决类不平衡问题，其中最流行的是过采样技术（如SMOTE）。这些方法在少数类中生成合成实例，以平衡数据集，执行数据扩充，从而提高预测机器学习（ML）模型的性能。在本文中，我们提出了一种新的数据扩充方法（改编自可解释人工智能），该方法在少数类中生成合成的反事实实例。与其他过采样技术不同，该方法使用实际特征值，而不是实例之间的插值，自适应地组合数据集中的现有过采样实例。报告了使用四种不同分类器和25个数据集的若干实验，结果表明，这种反事实增强方法（CFA）在少数类中生成了有用的合成数据点。实验还表明，CFA与许多其他过采样方法具有竞争性，其中许多是SMOTE的变体。讨论了CFAs性能的基础，以及在未来测试中可能表现更好或更差的条件。 摘要：Learning from class imbalanced datasets poses challenges for many machine learning algorithms. Many real-world domains are, by definition, class imbalanced by virtue of having a majority class that naturally has many more instances than its minority class (e.g. genuine bank transactions occur much more often than fraudulent ones). Many methods have been proposed to solve the class imbalance problem, among the most popular being oversampling techniques (such as SMOTE). These methods generate synthetic instances in the minority class, to balance the dataset, performing data augmentations that improve the performance of predictive machine learning (ML) models. In this paper we advance a novel data augmentation method (adapted from eXplainable AI), that generates synthetic, counterfactual instances in the minority class. Unlike other oversampling techniques, this method adaptively combines exist-ing instances from the dataset, using actual feature-values rather than interpolating values between instances. Several experiments using four different classifiers and 25 datasets are reported, which show that this Counterfactual Augmentation method (CFA) generates useful synthetic data points in the minority class. The experiments also show that CFA is competitive with many other oversampling methods many of which are variants of SMOTE. The basis for CFAs performance is discussed, along with the conditions under which it is likely to perform better or worse in future tests.
【5】 SocialVec: Social Entity Embeddings 标题：SocialVec：社会实体嵌入 链接：https://arxiv.org/abs/2111.03514
作者：Nir Lotan,Einat Minkov 机构：University of Haifa 摘要：本文介绍了SocialVec，一个从社交网络中获取社会世界知识的通用框架，并将该框架应用于Twitter。SocialVec学习流行账户的低维嵌入，这些账户代表普遍感兴趣的实体，基于其在账户内的共同出现模式以及个人用户，从而在社会人口统计学方面建模实体相似性。类似于单词嵌入，它促进了涉及文本处理的任务，我们期望社会实体嵌入有利于具有社会风味的任务。我们从一个包含130多万用户和他们关注的账户的Twitter网络样本中了解了大约200000个流行账户的社交嵌入，并评估了两个不同任务的嵌入结果。第一项任务涉及从用户的社交媒体档案中自动推断用户的个人特征。在另一项研究中，我们利用SocialSec嵌入来衡量推特新闻来源的政治偏见。在这两种情况下，我们证明了SocialVec嵌入与现有的实体嵌入方案相比具有优势。我们将公开SocialSec实体嵌入，以支持进一步探索Twitter中反映的社会世界知识。 摘要：This paper introduces SocialVec, a general framework for eliciting social world knowledge from social networks, and applies this framework to Twitter. SocialVec learns low-dimensional embeddings of popular accounts, which represent entities of general interest, based on their co-occurrences patterns within the accounts followed by individual users, thus modeling entity similarity in socio-demographic terms. Similar to word embeddings, which facilitate tasks that involve text processing, we expect social entity embeddings to benefit tasks of social flavor. We have learned social embeddings for roughly 200,000 popular accounts from a sample of the Twitter network that includes more than 1.3 million users and the accounts that they follow, and evaluate the resulting embeddings on two different tasks. The first task involves the automatic inference of personal traits of users from their social media profiles. In another study, we exploit SocialVec embeddings for gauging the political bias of news sources in Twitter. In both cases, we prove SocialVec embeddings to be advantageous compared with existing entity embedding schemes. We will make the SocialVec entity embeddings publicly available to support further exploration of social world knowledge as reflected in Twitter.
【6】 Dual Parameterization of Sparse Variational Gaussian Processes 标题：稀疏变分高斯过程的对偶参数化 链接：https://arxiv.org/abs/2111.03412
作者：Vincent Adam,Paul E. Chang,Mohammad Emtiyaz Khan,Arno Solin 机构：Aalto University Secondmind.ai, Espoo, Finland Cambridge, UK, RIKEN Center for AI Project, Tokyo, Japan 备注：To appear in Advances in Neural Information Processing Systems (NeurIPS 2021) 摘要：稀疏变分高斯过程（SVGP）方法是非共轭高斯过程推理的常用方法，因为它们具有计算优势。在本文中，我们通过使用双重参数化来提高它们的计算效率，其中每个数据示例被分配双重参数，类似于期望传播中使用的站点参数。我们的双重参数化使用自然梯度下降加速推理，并为超参数学习提供更严格的证据下限。该方法与当前的SVGP方法具有相同的内存开销，但它更快、更准确。 摘要：Sparse variational Gaussian process (SVGP) methods are a common choice for non-conjugate Gaussian process inference because of their computational benefits. In this paper, we improve their computational efficiency by using a dual parameterization where each data example is assigned dual parameters, similarly to site parameters used in expectation propagation. Our dual parameterization speeds-up inference using natural gradient descent, and provides a tighter evidence lower bound for hyperparameter learning. The approach has the same memory cost as the current SVGP methods, but it is faster and more accurate.
【7】 LILA: Language-Informed Latent Actions 标题：莱拉：语言知晓的潜在行动 链接：https://arxiv.org/abs/2111.03205
作者：Siddharth Karamcheti,Megha Srivastava,Percy Liang,Dorsa Sadigh 机构：Department of Computer Science, Stanford University 备注：Accepted at the 5th Conference on Robot Learning (CoRL). Joint first authorship. 21 Pages, 11 Figures 摘要：我们介绍了语言信息潜在动作（LILA），一个在人-机器人协作环境中学习自然语言界面的框架。LILA属于共享自治模式：除了提供离散的语言输入外，人类还获得了一个低维控制器$-$，例如，一个2自由度（DoF）操纵杆，可以左/右和上/下移动$-$来操作机器人。LILA学习使用语言来调节该控制器，为用户提供一个语言控制空间：如果给出类似“将谷物碗放在托盘上”的指令，LILA可以学习一个2-DoF空间，其中一维控制机器人末端执行器到碗的距离，另一个维度控制机器人末端执行器相对于碗上抓取点的姿势。我们通过现实世界的用户研究来评估LILA，用户可以在操作7自由度Franka-Emika熊猫手臂以完成一系列复杂操作任务的同时提供语言指导。我们表明，LILA模型不仅比模仿学习和末端效应器控制基线更具样本效率和性能，而且在质量上也受到用户的青睐。 摘要：We introduce Language-Informed Latent Actions (LILA), a framework for learning natural language interfaces in the context of human-robot collaboration. LILA falls under the shared autonomy paradigm: in addition to providing discrete language inputs, humans are given a low-dimensional controller $-$ e.g., a 2 degree-of-freedom (DoF) joystick that can move left/right and up/down $-$ for operating the robot. LILA learns to use language to modulate this controller, providing users with a language-informed control space: given an instruction like "place the cereal bowl on the tray," LILA may learn a 2-DoF space where one dimension controls the distance from the robot's end-effector to the bowl, and the other dimension controls the robot's end-effector pose relative to the grasp point on the bowl. We evaluate LILA with real-world user studies, where users can provide a language instruction while operating a 7-DoF Franka Emika Panda Arm to complete a series of complex manipulation tasks. We show that LILA models are not only more sample efficient and performant than imitation learning and end-effector control baselines, but that they are also qualitatively preferred by users.
【8】 On the Complexity of Dynamic Submodular Maximization 标题：关于动态子模极大化的复杂性 链接：https://arxiv.org/abs/2111.03198
作者：Xi Chen,Binghui Peng 摘要：我们研究了在$n$插入和删除流上最大化单调子模函数问题的动态算法。我们证明，对于任何常数$epsilon>0$，在基数约束下保持$（0.5 epsilon）$-近似解的任何算法，其摊销查询复杂度必须为$n$中的$mathit{polymone}$。此外，为了维持一个$0.584$的近似解，需要一个线性摊销查询复杂度。这与[LMNF 20，Mon20]的最新动态算法形成了鲜明对比，后者通过$mathsf{poly}log（n）$摊销查询复杂度实现了$（0.5-epsilon）$近似。从积极的方面看，当流仅为插入时，我们在基数约束和拟阵约束下分别给出了有效的算法，近似保证为$1-1/e-epsilon$和摊销查询复杂性$smash{O（log（k/epsilon）/epsilon^2）}$和$smash{k^{tilde{O}（1/epsilon 2）}logn}$，其中，$k$表示基数参数或拟阵的秩。 摘要：We study dynamic algorithms for the problem of maximizing a monotone submodular function over a stream of $n$ insertions and deletions. We show that any algorithm that maintains a $(0.5 epsilon)$-approximate solution under a cardinality constraint, for any constant $epsilon>0$, must have an amortized query complexity that is $mathit{polynomial}$ in $n$. Moreover, a linear amortized query complexity is needed in order to maintain a $0.584$-approximate solution. This is in sharp contrast with recent dynamic algorithms of [LMNF 20, Mon20] that achieve $(0.5-epsilon)$-approximation with a $mathsf{poly}log(n)$ amortized query complexity. On the positive side, when the stream is insertion-only, we present efficient algorithms for the problem under a cardinality constraint and under a matroid constraint with approximation guarantee $1-1/e-epsilon$ and amortized query complexities $smash{O(log (k/epsilon)/epsilon^2)}$ and $smash{k^{tilde{O}(1/epsilon^2)}log n}$, respectively, where $k$ denotes the cardinality parameter or the rank of the matroid.
【9】 Are You Smarter Than a Random Expert? The Robust Aggregation of Substitutable Signals 标题：你比随机专家还聪明吗？可替代信号的鲁棒聚集 链接：https://arxiv.org/abs/2111.03153
作者：Eric Neyman,Tim Roughgarden 备注：22 pages, 1 figure 摘要：整合专家预测的问题在机器学习、经济学、气候科学和国家安全等领域无处不在。尽管如此，我们对这个问题的理论理解还是相当肤浅的。本文在从广泛的信息结构中逆向选择专家知识的背景下，发起预测聚合的研究。虽然在一般情况下，不可能实现非平凡的性能保证，但我们表明，在专家信息结构的条件下（我们称之为投影替代物），实现非平凡的性能保证是可能的。投射替代条件是信息替代的概念：学习专家信号的边际收益递减。我们表明，在投影替代条件下，对专家的预测进行平均大大改进了信任随机专家的策略。然后，我们考虑一个更宽松的设置，其中聚合器可以访问之前。我们表明，通过对专家的预测进行平均，然后通过将平均值从先验值移开一个常数因子来对平均值进行极值化，聚合器的性能保证大大优于不知道先验值的情况下的性能保证。我们的结果为以往的极值化实证研究提供了理论基础，并有助于指导极值化的适当数量。 摘要：The problem of aggregating expert forecasts is ubiquitous in fields as wide-ranging as machine learning, economics, climate science, and national security. Despite this, our theoretical understanding of this question is fairly shallow. This paper initiates the study of forecast aggregation in a context where experts' knowledge is chosen adversarially from a broad class of information structures. While in full generality it is impossible to achieve a nontrivial performance guarantee, we show that doing so is possible under a condition on the experts' information structure that we call emph{projective substitutes}. The projective substitutes condition is a notion of informational substitutes: that there are diminishing marginal returns to learning the experts' signals. We show that under the projective substitutes condition, taking the average of the experts' forecasts improves substantially upon the strategy of trusting a random expert. We then consider a more permissive setting, in which the aggregator has access to the prior. We show that by averaging the experts' forecasts and then emph{extremizing} the average by moving it away from the prior by a constant factor, the aggregator's performance guarantee is substantially better than is possible without knowledge of the prior. Our results give a theoretical grounding to past empirical research on extremization and help give guidance on the appropriate amount to extremize.
【10】 Scaffolding Sets 标题：脚手架套装 链接：https://arxiv.org/abs/2111.03135
作者：Maya Burhanpurkar,Zhun Deng,Cynthia Dwork,Linjun Zhang 机构：†Harvard University 备注：32 pages, 4 figures 摘要：预测器将总体中的单个实例映射到区间$[0,1]$。对于总体子集的集合$mathcal C$，如果在$mathcal C$中的每个集合上同时校准预测值，则预测值相对于$mathcal C$进行多重校准。我们开始研究脚手架集的构造，脚手架集是集的一个小集合$mathcal S$，其属性是，关于$mathcal S$的多重校准确保了预测值的正确性，而不仅仅是校准。我们的方法受到民间智慧的启发，即神经网络的中间层学习高度结构化和有用的数据表示。 摘要：Predictors map individual instances in a population to the interval $[0,1]$. For a collection $mathcal C$ of subsets of a population, a predictor is multi-calibrated with respect to $mathcal C$ if it is simultaneously calibrated on each set in $mathcal C$. We initiate the study of the construction of scaffolding sets, a small collection $mathcal S$ of sets with the property that multi-calibration with respect to $mathcal S$ ensures correctness, and not just calibration, of the predictor. Our approach is inspired by the folk wisdom that the intermediate layers of a neural net learn a highly structured and useful data representation.
【11】 Successor Feature Neural Episodic Control 标题：后继特征神经事件控制 链接：https://arxiv.org/abs/2111.03110
作者：David Emukpere,Xavier Alameda-Pineda,Chris Reinke 机构：RobotLearn, INRIA Grenoble, LJK, UGA 摘要：强化学习的一个长期目标是构建智能代理，以显示快速学习和类似于人类和动物的灵活技能转移。本文研究了两个框架的集成来实现这些目标：情景控制和后续特征。情景控制是一种基于认知的方法，依赖于情景记忆，一种基于实例的agent经验记忆模型。同时，后继特征和广义策略改进（SF&GPI）是一个元学习和迁移学习框架，允许学习任务的策略，这些策略可以有效地用于具有不同奖励函数的后续任务。单独而言，这两种技术在极大地提高样本效率和优雅地重用以前学习的策略方面已经显示出令人印象深刻的结果。因此，我们在一个单一的强化学习框架中概述了这两种方法的组合，并实证说明了其好处。 摘要：A longstanding goal in reinforcement learning is to build intelligent agents that show fast learning and a flexible transfer of skills akin to humans and animals. This paper investigates the integration of two frameworks for tackling those goals: episodic control and successor features. Episodic control is a cognitively inspired approach relying on episodic memory, an instance-based memory model of an agent's experiences. Meanwhile, successor features and generalized policy improvement (SF&GPI) is a meta and transfer learning framework allowing to learn policies for tasks that can be efficiently reused for later tasks which have a different reward function. Individually, these two techniques have shown impressive results in vastly improving sample efficiency and the elegant reuse of previously learned policies. Thus, we outline a combination of both approaches in a single reinforcement learning framework and empirically illustrate its benefits.
【12】 PerSpeechNorm: A Persian Toolkit for Speech Processing Normalization 标题：PerSpeechNorm：一个用于语音处理标准化的波斯语工具包 链接：https://arxiv.org/abs/2111.03470
作者：Romina Oji,Seyedeh Fatemeh Razavi,Sajjad Abdi Dehsorkh,Alireza Hariri,Hadi Asheri,Reshad Hosseini 机构：∗ School of ECE, College of Engineering, University of Tehran, Tehran, Iran, † Hoosh Afzare Rahbare Aryaman (HARA) Company, Tehran, Iran, ‡Equal contribution 摘要：通常，语音处理模型包括语言模型和声学模型。不管语言模型的复杂性和变体如何，语言模型中都需要三个关键的预处理步骤：清理、规范化和标记化。在上述步骤中，规范化步骤对于纯文本应用程序中的格式统一非常重要。然而，对于语音处理模块中的嵌入式语言模型，规范化并不局限于格式统一。此外，它必须将每个可读符号、数字等转换为它们的发音方式。据我们所知，语音处理模块中没有用于嵌入式语言模型的波斯语规范化工具包，因此在本文中，我们提出了一种用于语音应用中文本处理的开源规范化工具包。简单地说，我们考虑不同的可读波斯文本，如符号（通用货币，α，@，URL等），数字（日期，时间，电话号码，国家代码等），等等。与其他可用的波斯语文本规范化工具的比较表明了该方法在语音处理中的优越性。此外，将所提出的函数之一（句子分离）的模型性能与其他常用自然语言库（如HAZM和Parsivar）进行比较，表明所提出的方法具有适当的性能。此外，对一些波斯维基百科数据的评估也证实了该方法的正确性能。 摘要：In general, speech processing models consist of a language model along with an acoustic model. Regardless of the language model's complexity and variants, three critical pre-processing steps are needed in language models: cleaning, normalization, and tokenization. Among mentioned steps, the normalization step is so essential to format unification in pure textual applications. However, for embedded language models in speech processing modules, normalization is not limited to format unification. Moreover, it has to convert each readable symbol, number, etc., to how they are pronounced. To the best of our knowledge, there is no Persian normalization toolkits for embedded language models in speech processing modules, So in this paper, we propose an open-source normalization toolkit for text processing in speech applications. Briefly, we consider different readable Persian text like symbols (common currencies, #, @, URL, etc.), numbers (date, time, phone number, national code, etc.), and so on. Comparison with other available Persian textual normalization tools indicates the superiority of the proposed method in speech processing. Also, comparing the model's performance for one of the proposed functions (sentence separation) with other common natural language libraries such as HAZM and Parsivar indicates the proper performance of the proposed method. Besides, its evaluation of some Persian Wikipedia data confirms the proper performance of the proposed method.
【13】 Segmentation of 2D Brain MR Images 标题：二维脑MR图像的分割 链接：https://arxiv.org/abs/2111.03370
作者：Angad Ripudaman Singh Bajwa 机构： 2 1 Indian Institute of Technology (IIT), Varanasi (BHU) 2 National Institute of Technology (NIT) 摘要：脑肿瘤分割是医学图像处理中的一项重要任务。脑肿瘤的早期诊断在改善治疗可能性和提高患者生存率方面起着至关重要的作用。从大量MRI图像中手动分割用于癌症诊断的脑肿瘤是一项既困难又耗时的任务。有必要对脑肿瘤图像进行自动分割。本项目的目的是提供一种MRI图像的脑肿瘤自动分割方法，以帮助准确快速地定位肿瘤。 摘要：Brain tumour segmentation is an essential task in medical image processing. Early diagnosis of brain tumours plays a crucial role in improving treatment possibilities and increases the survival rate of the patients. Manual segmentation of the brain tumours for cancer diagnosis, from large number of MRI images, is both a difficult and time-consuming task. There is a need for automatic brain tumour image segmentation. The purpose of this project is to provide an automatic brain tumour segmentation method of MRI images to help locate the tumour accurately and quickly.
【14】 Maillard Sampling: Boltzmann Exploration Done Optimally 标题：美拉德抽样：博尔兹曼勘探做得最好 链接：https://arxiv.org/abs/2111.03290
作者：Jie Bian,Kwang-Sung Jun 机构：University of Arizona 摘要：Maillard（2013）的博士论文提出了一种用于$K$武装强盗问题的随机算法。这种鲜为人知的算法，我们称之为美拉德抽样（MS），以封闭形式计算选择每个手臂的概率，这有助于根据bandit记录的数据进行反事实评估，但Thompson抽样（业界广泛采用的bandit算法）缺乏这种算法。基于这样的优点，我们重新审视了MS，并进行了改进分析，以表明它同时实现了渐近最优性和$sqrt{KTlog{T}}$极大极小遗憾界，其中，$T$是时间范围，与标准渐近最优UCB的性能相匹配。然后，我们提出了一种称为MS$^ $的MS变体，它改进了它在$sqrt{KTlog{K}}}$上的极小极大值，而不丢失渐近最优性。MS$^ $还可以调整为具有攻击性（即更少的探索），而不会失去理论保证，这是现有bandit算法无法提供的独特功能。我们的数值评估显示了MS$^ $的有效性。 摘要：The PhD thesis of Maillard (2013) presents a randomized algorithm for the $K$-armed bandit problem. This less-known algorithm, which we call Maillard sampling (MS), computes the probability of choosing each arm in a closed form, which is useful for counterfactual evaluation from bandit-logged data but was lacking from Thompson sampling, a widely-adopted bandit algorithm in the industry. Motivated by such merit, we revisit MS and perform an improved analysis to show that it achieves both the asymptotical optimality and $sqrt{KTlog{T}}$ minimax regret bound where $T$ is the time horizon, which matches the standard asymptotically optimal UCB's performance. We then propose a variant of MS called MS$^ $ that improves its minimax bound to $sqrt{KTlog{K}}$ without losing the asymptotic optimality. MS$^ $ can also be tuned to be aggressive (i.e., less exploration) without losing theoretical guarantees, a unique feature unavailable from existing bandit algorithms. Our numerical evaluation shows the effectiveness of MS$^ $.
【15】 Big-Step-Little-Step: Efficient Gradient Methods for Objectives with Multiple Scales 标题：大步小步：多尺度目标的高效梯度法 链接：https://arxiv.org/abs/2111.03137
作者：Jonathan Kelner,Annie Marsden,Vatsal Sharan,Aaron Sidford,Gregory Valiant,Honglin Yuan 机构：MIT, Stanford University, USC† 备注：95 pages, 4 figures; authors are listed in alphabetical order 摘要：我们提供了新的基于梯度的方法来有效地解决一类广泛的病态优化问题。我们考虑函数F$:Mathbb{r} d dRealTraveMathbB{R}$的最小化问题，该问题可隐式分解为$M$未知的非交互光滑、强凸函数的总和，并提供了一种方法，该方法解决了该问题的一些梯度估计，其尺度（对数因子）。作为组件条件数平方根的乘积。这个复杂度界限（我们证明它几乎是最优的）可以比加速梯度法的复杂度界限几乎成倍地提高，加速梯度法的复杂度界限随着条件数$f$的平方根而增长。此外，我们提供了有效的方法来解决这个多尺度优化问题的随机二次变量。我们的方法没有学习$f$的分解（这将非常昂贵），而是采用标准方法的干净递归“大步小步”交错。由此产生的算法使用$tilde{mathcal{O}}（dm）$空间，在数值上是稳定的，并且为更细粒度地理解超出条件数的凸优化的复杂性打开了大门。 摘要：We provide new gradient-based methods for efficiently solving a broad class of ill-conditioned optimization problems. We consider the problem of minimizing a function $f : mathbb{R}^d rightarrow mathbb{R}$ which is implicitly decomposable as the sum of $m$ unknown non-interacting smooth, strongly convex functions and provide a method which solves this problem with a number of gradient evaluations that scales (up to logarithmic factors) as the product of the square-root of the condition numbers of the components. This complexity bound (which we prove is nearly optimal) can improve almost exponentially on that of accelerated gradient methods, which grow as the square root of the condition number of $f$. Additionally, we provide efficient methods for solving stochastic, quadratic variants of this multiscale optimization problem. Rather than learn the decomposition of $f$ (which would be prohibitively expensive), our methods apply a clean recursive "Big-Step-Little-Step" interleaving of standard methods. The resulting algorithms use $tilde{mathcal{O}}(d m)$ space, are numerically stable, and open the door to a more fine-grained understanding of the complexity of convex optimization beyond condition number.
【16】 Functional connectivity ensemble method to enhance BCI performance (FUCONE) 标题：增强脑-机接口性能的功能连通性集成方法(FUCONE) 链接：https://arxiv.org/abs/2111.03122
作者：Marie-Constance Corsi,Sylvain Chevallier,Fabrizio De Vico Fallani,Florian Yger 机构：Inria, Aramis project-team, Paris, France, Sorbonne Universit´e, Paris, France, Institut du Cerveau - Paris Brain Institute - ICM, Paris, France, Inserm, Paris, France, CNRS, Paris, France, AP-HP, Hˆopital de la Piti´e Salpˆetriere, Paris, France 摘要：功能连接性是研究大脑振荡活动的一个关键方法，它为神经元相互作用的潜在动力学提供了重要的见解，并且主要应用于大脑活动分析。基于脑-机接口信息几何学的进展，我们提出了一个新的框架，该框架结合了功能连通性估计器和基于协方差的管道来分类精神状态，如运动想象。为每个估计器训练一个黎曼分类器，集成分类器将每个特征空间中的决策组合起来。对功能连通性估计器进行了全面评估，并在不同条件和数据集上对性能最佳的管道FUCONE进行了评估。使用荟萃分析对数据集的结果进行汇总，FUCONE的表现明显优于所有最先进的方法。性能增益主要归因于特征空间的改进多样性，提高了集成分类器对主题间和主题内可变性的鲁棒性。 摘要：Functional connectivity is a key approach to investigate oscillatory activities of the brain that provides important insights on the underlying dynamic of neuronal interactions and that is mostly applied for brain activity analysis. Building on the advances in information geometry for brain-computer interface, we propose a novel framework that combines functional connectivity estimators and covariance-based pipelines to classify mental states, such as motor imagery. A Riemannian classifier is trained for each estimator and an ensemble classifier combines the decisions in each feature space. A thorough assessment of the functional connectivity estimators is provided and the best performing pipeline, called FUCONE, is evaluated on different conditions and datasets. Using a meta-analysis to aggregate results across datasets, FUCONE performed significantly better than all state-of-the-art methods. The performance gain is mostly imputable to the improved diversity of the feature spaces, increasing the robustness of the ensemble classifier with respect to the inter- and intra-subject variability.
