In its year #1, Generative AI (GenAI) tackled security use cases individually including threat detection, alerts investigation, security posture analysis, and some others. As we enter year #2, this session explores the integration of these use cases and more through a consolidated GenAI architecture seamlessly integrated into any product. Discover how this consolidation not only enhances cyber security across existing infrastructure, but also boosts the effectiveness of security teams in their daily operations.

Factual questions typically can be answered correctly at different levels of granularity. For example, both ``August 4, 1961'' and ``1961'' are correct answers to the question ``When was Barack Obama born?''. Standard question answering (QA) evaluation protocols, however, do not explicitly take this into account and compare a predicted answer against answers of a single granularity level. I will present recent work that tries to bridge this gap with a new evaluation setting, where a predicted answer is evaluated in terms of accuracy and informativeness against a set of multi-granularity answers. Our experiments show that large language models with standard decoding tend to generate specific answers, which are often incorrect. In contrast, when evaluated on multi-granularity answers, a custom decoding algorithm geared towards answering at an appropriate level of granularity yields a nearly 20 point increase in accuracy (and more for rare entities). Overall, this reveals that standard evaluation and decoding schemes may significantly underestimate the knowledge encapsulated in LMs.

Large Language Models (LLMs) are essential in enhancing productivity across industries through natural language. However, their effectiveness is often limited by the extent of their training data, resulting in poor performance when dealing with real-time events and new knowledge the LLM isn’t trained on. Retrieval Augmented Generation (RAG) solves these problems by augmenting LLMs with an external knowledge base. This enhances the quality of their response, improving accuracy, and reducing hallucinations. RAG not only extends the utility of LLMs but also provides a cheaper alternative to time-consuming re-training runs. Deploying and scaling RAG applications comes with its own set of challenges. I will present techniques for end-to-end RAG deployment, as well as highlight usage of NVIDIA TensorRT-LLM for this purpose

Text-to-image generative diffusion models have recently risen to prominence, achieving unprecedented success and popularity. These text-to-image models are trained on huge amounts of web-scraped image-caption pairs. As a result, these models encode real-world information and correlations, such as the identity of the President of the United States, or the color of the sky. While this knowledge can be useful, and allows easy and efficient generation of beautiful images from simple prompts, it may also be outdated, reflect assumptions and biases (e.g., doctors are always white male), or violate copyrights (as was demonstrated in recent lawsuits for models imitating artistic styles). However, model providers and creators currently have no efficient means to update models without either retraining them---which is costly in computation and time, and might also require data curation---or requiring explicit prompt engineering from the end user.
In this talk, we will discuss three of our recent papers, which aim to offer a fast and practical way to control model behavior post-training. We modify a small, targeted part of the model that is responsible for encoding a certain part in the computation process of the deep network. This is done without training, by editing the model weights using a closed-form solution. The different papers target different parts of the model and various types of information encoded in it: implicit assumptions, factual associations, artistic style, biases, and harmful content. Overall, the methods presented in the talk offer a fast and practical means for safe deployment of text-to-image models.
 

The primary focus of local news websites is to deliver news relevant to the local community. Taboola’s  personalized recommender system, aims to determine whether a potential item contains local or global information. Typically, the item's text can be used to extract the location using a standard Named Entity Recognition model. However, difficulties arise when the location is implicitly referenced in the title, such as mentioning a local sports team or a prominent individual's name. Standard Named Entity Recognition models struggle to identify implicit locations, whereas ChatGPT excels at this task. To explore the potential benefits, we used ChatGPT to enhance the localization coverage of our news items, in order to provide this valuable information to our recommender system model.

Sharing the insights from Microsoft's massive scale experiment in introducing 20+ AI Assistants into key products (from Office to Windows) and what has been learned as early adopters in the conversational-driven UX domain.

In the midst of the war between Israel and Hamas, Israel is also engaged in an intense online battle over its ethical reputation. Given the scope and scale of anti-semitic, hateful, and often misleading/fake content, Rhetoric AI developed a persuasive argumentation platform to facilitate an effective response. We leverage foundation models to understand the content of posts, identify whether they violated terms of service, and systematically counter-argue prevalent anti-Israel content online. We leverage principles from areas such as Argument Mining in NLP and Rhetorical Style from collegiate debate in order to imbue a language model agent with persuasive argumentation capabilities. This talk aims to share insights, best-practices, and methods for how to build AI Agents for Persuasive Argumentation. 

By 2026, 90% of media online will be AI generated. Much of it hyperrealistic: the type where we can't tell if it's real or fake. Every industry will be impacted, and every organization disrupted. Adaptation is key, and new security systems will be introduced to identify what's real and what's fake. But how will it work? Let's dive in. 

Diffusion models constitute the leading generative approach for the visual world. This technology keeps wowing the world with new capabilities almost on a weekly basis, with newer and larger models being constantly released. These models typically are trained with a lot of data, for a lot of time, and encapsulate a great deal of knowledge about the world. At the same time, since these models are so expressive, it is difficult to get a specifically desired result. 

In this talk, I present some of my lab's latest work, and discuss how large scale models can be controlled and results edited with minimal resources. I portray a few of our ideas and approaches, requiring varying amounts of computation or data, over the natural images and 3D human animation domains.

Diffusion models have revolutionized image generation and editing, producing state-of-the-art results in conditioned and unconditioned image synthesis. While current techniques enable user control over the degree of change in an image edit, the controllability is limited to global changes over an entire edited region. This paper introduces a novel framework that enables customization of the amount of change per pixel or per image region. Our framework can be integrated into any existing diffusion model, enhancing it with this capability. Such granular control on the quantity of change opens up a diverse array of new editing capabilities, such as control of the extent to which individual objects are modified, or the ability to introduce gradual spatial changes. Furthermore, we showcase the framework’s effectiveness in soft-inpainting—the completion of portions of an image while subtly adjusting the surrounding areas to ensure seamless integration. Additionally, we introduce a new tool for exploring the effects of different change quantities. Our framework operates solely during inference, requiring no model training or fine-tuning. We demonstrate our method with the current open state-of-the-art models, and validate it via both quantitative and qualitative comparisons, and a user study. Our code will be available upon publication.

Iron Swords War is one of the first wars where critical visual information is found on millions of social network videos released daily. Israel was caught unprepared and had difficulties dealing with the huge video data volumes. Visual Layer is a startup creating & maintaining the popular open source fastdup for managing large scale visual data in the context of data quality issues. In this talk I will present the pro bono work Visual Layer did for the Ministry of Diaspora and Ministry of Defense for building a video collection for managing, exploring and visualizing war crime videos.  In a few days we have adapted our cloud infrastructure to support data analyst video search & exploration use cases. Our work was featured in the Israeli Parliament at the Defense and Foreign Affairs committee as an example for a successful government and industry collaboration. Joint work with Matrix Defense. 

Deep learning is delivering unprecedented performance when applied to various data modalities, yet there are data distributions over which it utterly fails. The question of what makes a data distribution suitable for deep learning is a fundamental open problem in the field.  In this talk I will present a recent theory aiming to address the problem via tools from quantum physics.  The theory establishes that certain neural networks are capable of accurate prediction over a data distribution if and only if the data distribution admits low quantum entanglement under certain partitions of features.  This brings forth practical methods for adaptation of data to neural networks, and vice versa.  Experiments with widespread models over various datasets will demonstrate the findings.  An underlying theme of the talk will be the potential of physics to advance our understanding of the relation between deep learning and real-world data.

The talk is based on NeurIPS 2023 papers co-authored with my students Noam Razin, Yotam Alexander, Nimrod De La Vega and Tom Verbin.
 

Robust sequential decision making offers a way to overcome deviations of the model caused by having access to finite data, disturbances, and model drifts. In this talk I will describe several recent and classical results in the field and outline the limitations as well as some potential research directions. The talk does not assume any prior knowledge and is meant to give a broad overview of the field. 

Machine learning (ML) is integrated into our society, it is present in the judicial, health, transportation, and financial systems. As its integration increases, the necessity of ML transparency increases. The fields of explainable and interpretable ML attempt to add transparency to ML— either by adding explanations to a given black-box ML model or by constructing a model that is inherently interpretable and self-explanatory.

Despite the importance of explainability and interpretability, their foundations remain elusive. Fundamental questions are left unanswered: How to define explainability and interpretability? Is there a tradeoff between performance and interpretability? How to evaluate the quality of explanation? In this talk, we demonstrate how some of these fundamental questions can be addressed by focusing on a specific type of explanation—concept explanation. We introduce a new axiomatic approach to define this type of explanation and empirically examine the benefits of the new definition for model improvement.
 

The seminal Fast Johnson-Lindenstrauss (Fast JL) transform by Ailon and Chazelle (SICOMP’09) embeds a set of n points in d-dimensional Euclidean space into optimal 𝑘 = 𝑂(𝜀^{−2} 𝑙𝑛 𝑛) dimensions, while preserving all pairwise distances to within a factor (1 ± ε). The Fast JL transform supports computing the embedding of a data point in 𝑂(𝑑 𝑙𝑛 𝑑+𝑘 𝑙𝑛^2 𝑛) time, where the (𝑑 𝑙𝑛 𝑑) term comes from multiplication with a d×d Hadamard matrix and the (𝑘 𝑙𝑛^2 𝑛) term comes from multiplication with a sparse k×d matrix.
Despite the Fast JL transform being more than a decade old, it is one of the fastest dimensionality reduction techniques for many tradeoffs between ε, d and n.
In this work, we give a surprising new analysis of the Fast JL transform, showing that the 𝑘 𝑙𝑛^2 𝑛 term in the embedding time can be improved to (𝑘 𝑙𝑛 𝑛). The improvement follows by using an even sparser matrix. We also complement our improved analysis with a lower bound showing that our new analysis is in fact tight.

The AI-powered bank: Numerous use cases are emerging to achieve efficiencies, accelerate productivity, and reimagine customer experiences. 

Helping sellers price their listings is an important and challenging task for E-commerce marketplaces. To help the seller gain trust in the recommended price, a collection of supporting similar listings are retrieved and provided along with their prices. We address the problem of retrieval-based price recommendation using a novel approach, which enables a trade-off adjustment between semantic similarity and price accuracy. Balancing the two required since, based on our study, retrieval of semantically similar listings does not guarantee pricing accuracy. In contrast, a price-accuracy driven approach may produce less semantically supporting listings. We also suggest a third method - training a Multi-Task network which learns in parallel both semantic similarity and a pricing-based objective. Framing the solution as a Multi-Task network unfolds the ability to control the balance between explainability and accuracy, thus providing a powerful tool to precisely tailor the correct pricing solution to different real world business use cases.

This talk will explore how Artificial Intelligence (AI) is changing education, focusing on building resilience for the future. We are moving from big ideas about AI to actually using it in schools. This session will look at how AI can make learning better and help education systems deal with new challenges. We'll discuss the benefits of AI in creating personalized learning experiences and supporting teachers and students. But it's not all easy - we also need to think about important issues like using AI responsibly, protecting privacy, and making sure everyone has equal access to these technologies. Our goal is to understand how education can use AI in a smart and fair way, helping to create a strong and adaptable learning environment. Join us to see how AI can help shape a strong and innovative future for education.

This talk will present AI tools and methods to expose inauthentic hostile campaigns in social networks that aim to harm national resilience. 

Real examples will be shown from the last few months.
 

The coordinated terrorist attacks on October 7, 2023, resulted in catastrophic atrocities, and marked the beginning of the 2023 Israel–Hamas war. The overwhelming coverage by mainstream and social media, characterized by extreme details and graphic images, vividly transported viewers to the horrifying scene. It remains unclear to what extent such indirect exposure influences the occurrence of stress, anxiety, and post-traumatic symptoms. We analyzed data from a three-year prospective study in which 4,797 participants received smartwatches and completed daily questionnaires, supplemented by a nationwide clinical survey with 2,536 participants. Among the participants not directly exposed, we estimated PTSD prevalence to be 22.9-36.0% and moderate to severe anxiety prevalence to be 22.9-55.32%, with 752,057 daily questionnaires before and after October 7 further indicating higher stress levels than those reported in previous events, including political disputes, the COVID-19 pandemic, and past armed conflicts. The occurrences of PTSD and anxiety are well explained by increased and persistent news consumption, and especially by the availability of gory videos on social media. Continuous monitoring of participants via smartwatches and daily questionnaires further revealed considerable differences in stress, mood, step counts, sleep quality, and duration in the first week after the October 7 events among those who later developed PTSD. This study demonstrates the unprecedented amplifying effect of mass media on mental health in terror and war settings and highlights the potential of continuous monitoring for early detection and prompt treatment of those in need.