Gargi Balasubramaniam

Domain generalization aims to learn a model over multiple training environments to generalize to unseen environments. Recently, Wang et al. [2022] proposed Invariant-feature Subspace Recovery (ISR), a domain generalization algorithm that uses the means of class-conditional data distributions to provably identify the invariant-feature subspace under a given causal model. However, due to the specific assumptions of the causal model, the original ISR algorithm is conditioned on a single class only, without utilizing information from the rest of the classes. In this work, we consider the setting of multi-class classification under a more general causal model, and propose an extension of the ISR algorithm, called ISR-Multiclass. This proposed algorithm can provably recover the invariant-feature subspace with ⌈d_spu/k ⌉+ 1 environments, where d_spu is the number of spurious features and k is the number of classes. Empirically, we first examine ISR-Multiclass in a synthetic dataset, and demonstrate its superiority over the original ISR in the multi-class setting. Furthermore, we conduct experiments in Multiclass Coloured MNIST, a semi-synthetic dataset with strong spurious correlations, and show that ISR-Multiclass can significantly improve the robustness of neural nets trained by various methods (e.g., ERM and IRM) against spurious correlations.

Multimodal learning considers learning from multi-modality data, aiming to fuse heterogeneous sources of information. However, it is not always feasible to leverage all available modalities due to memory constraints. Further, training on all the modalities may be inefficient when redundant information exists within data, such as different subsets of modalities providing similar performance. In light of these challenges, we study modality selection, intending to efficiently select the most informative and complementary modalities under certain computational constraints. We formulate a theoretical framework for optimizing modality selection in multimodal learning and introduce a utility measure to quantify the benefit of selecting a modality. For this optimization problem, we present efficient algorithms when the utility measure exhibits monotonicity and approximate submodularity. We also establish a novel correspondence between the utility measure and existing marginal contribution scores based on the Shapely value. Last, we demonstrate the efficacy of our algorithm on synthetic (Patch-MNIST) and real-world (PEMS-SF Traffic) datasets.

We present an empirical analysis of different domain generalization algorithms - a setting where the learner has access to several domains, and can leverage these data points to best generalize to new, unseen domains. In this work, we present a structured approach to compare and contrast representations learnt by these methods, both qualitatively and quantitatively. We enable this analysis by treating each algorithm as a descriptor producing oracle. Our work is evaluated on the PACS dataset - containing images in the styles of Photos, Art Paintings, Cartoon and Sketch. We propose the following metrics and their motivations: a) Test-time performance and fairness evaluated by accuracy at 3 levels of granularity. b) Interpretability and Reliability via calibration Analysis c) Domain Invariance via qualitative TSNE Visualization.

Image similarity estimation and copy detection are upcoming research areas. Given today’s era of social interaction - social media serves as a ground for spread of misinformation, with manipulated images being a common modality. In order to tackle this issue, image matching is a key component of content tracing systems which are used to flag manipulated and violating content. Furthermore, a wide variety of trans- formations are possible by using advanced and accessible image editing software - which implies that this is not a trivial problem. In this context, we aim to solve the following problem: can we determine whether an image is an edited copy of another image? We take inspiration from the Image Similarity Challenge 2021conducted at CVPR ’21 and NeurIPS ’21. Specifically, given a query image (which may be manipulated) - we aim to develop a model that can predict whether this image is a derived from a large set of reference images, and find the corresponding matching image from this reference set. Our approach to achieve this goal is to learn reliable feature descriptors for each image, which can be used to obtain the most similar images from the reference set.

From the point of view of a software engineer, having safe and optimal controllers for real life systems like cyber physical systems is a crucial requirement before deployment. Given the mathematical model of these systems along with their specifications, model checkers can be used to synthesize controllers for them.The given work proposes novel approaches for making controller analysis easier by using machine learning to represent the controllers synthesized by model checkers in a succinct manner, while also incorporating the domain knowledge of the system. It also proposes the implementation of a visualization tool which will be integrated into existing model checkers. A lucid controller representation along with a tool to visualize it will help the software engineer debug and monitor the system much more efficiently.