Gargi Balasubramaniam

Domain generalization asks for models trained over a set of training environments to generalize well in unseen test environments. Recently, a series of algorithms such as Invariant Risk Minimization (IRM) have been proposed for domain generalization. However, Rosenfeld et al. (2021) shows that in a simple linear data model, even if non-convexity issues are ignored, IRM and its extensions cannot generalize to unseen environments with less than ds+1 training environments, where ds is the dimension of the spurious-feature subspace. In this work, we propose Invariant-feature Subspace Recovery (ISR): a new class of algorithms to achieve provable domain generalization across the settings of classification and regression problems. First, in the binary classification setup of Rosenfeld et al. (2021), we show that our first algorithm, ISR-Mean, can identify the subspace spanned by invariant features from the first-order moments of the class-conditional distributions, and achieve provable domain generalization with ds+1 training environments. Our second algorithm, ISR-Cov, further reduces the required number of training environments to O(1) using the information of second-order moments. Notably, unlike IRM, our algorithms bypass non-convexity issues and enjoy global convergence guarantees. Next, we extend ISR-Mean to the more general setting of multi-class classification and propose ISR-Multiclass, which leverages class information and provably recovers the invariant-feature subspace with ⌈ds/k⌉+1 training environments for k-class classification. Finally, for regression problems, we propose ISR-Regression that can identify the invariant-feature subspace with ds+1 training environments. Empirically, we demonstrate the superior performance of our ISRs on synthetic benchmarks. Further, ISR can be used as post-processing methods for feature extractors such as neural nets.

Domain generalization aims to learn a model over multiple training environments to generalize to unseen environments. Recently, Wang et al. [2022] proposed Invariant-feature Subspace Recovery (ISR), a domain generalization algorithm that uses the means of class-conditional data distributions to provably identify the invariant-feature subspace under a given causal model. However, due to the specific assumptions of the causal model, the original ISR algorithm is conditioned on a single class only, without utilizing information from the rest of the classes. In this work, we consider the setting of multi-class classification under a more general causal model, and propose an extension of the ISR algorithm, called ISR-Multiclass. This proposed algorithm can provably recover the invariant-feature subspace with ⌈d_spu/k ⌉+ 1 environments, where d_spu is the number of spurious features and k is the number of classes. Empirically, we first examine ISR-Multiclass in a synthetic dataset, and demonstrate its superiority over the original ISR in the multi-class setting. Furthermore, we conduct experiments in Multiclass Coloured MNIST, a semi-synthetic dataset with strong spurious correlations, and show that ISR-Multiclass can significantly improve the robustness of neural nets trained by various methods (e.g., ERM and IRM) against spurious correlations.

Multimodal learning considers learning from multi-modality data, aiming to fuse heterogeneous sources of information. However, it is not always feasible to leverage all available modalities due to memory constraints. Further, training on all the modalities may be inefficient when redundant information exists within data, such as different subsets of modalities providing similar performance. In light of these challenges, we study modality selection, intending to efficiently select the most informative and complementary modalities under certain computational constraints. We formulate a theoretical framework for optimizing modality selection in multimodal learning and introduce a utility measure to quantify the benefit of selecting a modality. For this optimization problem, we present efficient algorithms when the utility measure exhibits monotonicity and approximate submodularity. We also establish a novel correspondence between the utility measure and existing marginal contribution scores based on the Shapely value. Last, we demonstrate the efficacy of our algorithm on synthetic (Patch-MNIST) and real-world (PEMS-SF Traffic) datasets.

We present an empirical analysis of different domain generalization algorithms - a setting where the learner has access to several domains, and can leverage these data points to best generalize to new, unseen domains. In this work, we present a structured approach to compare and contrast representations learnt by these methods, both qualitatively and quantitatively. We enable this analysis by treating each algorithm as a descriptor producing oracle. Our work is evaluated on the PACS dataset - containing images in the styles of Photos, Art Paintings, Cartoon and Sketch. We propose the following metrics and their motivations: a) Test-time performance and fairness evaluated by accuracy at 3 levels of granularity. b) Interpretability and Reliability via calibration Analysis c) Domain Invariance via qualitative TSNE Visualization.

Image similarity estimation and copy detection are upcoming research areas. Given today’s era of social interaction - social media serves as a ground for spread of misinformation, with manipulated images being a common modality. In order to tackle this issue, image matching is a key component of content tracing systems which are used to flag manipulated and violating content. Furthermore, a wide variety of trans- formations are possible by using advanced and accessible image editing software - which implies that this is not a trivial problem. In this context, we aim to solve the following problem: can we determine whether an image is an edited copy of another image? We take inspiration from the Image Similarity Challenge 2021conducted at CVPR ’21 and NeurIPS ’21. Specifically, given a query image (which may be manipulated) - we aim to develop a model that can predict whether this image is a derived from a large set of reference images, and find the corresponding matching image from this reference set. Our approach to achieve this goal is to learn reliable feature descriptors for each image, which can be used to obtain the most similar images from the reference set.

From the point of view of a software engineer, having safe and optimal controllers for real life systems like cyber physical systems is a crucial requirement before deployment. Given the mathematical model of these systems along with their specifications, model checkers can be used to synthesize controllers for them.The given work proposes novel approaches for making controller analysis easier by using machine learning to represent the controllers synthesized by model checkers in a succinct manner, while also incorporating the domain knowledge of the system. It also proposes the implementation of a visualization tool which will be integrated into existing model checkers. A lucid controller representation along with a tool to visualize it will help the software engineer debug and monitor the system much more efficiently.