{"id":851687,"date":"2022-06-13T04:11:27","date_gmt":"2022-06-13T11:11:27","guid":{"rendered":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/?post_type=msr-project&p=851687"},"modified":"2022-07-25T00:05:20","modified_gmt":"2022-07-25T07:05:20","slug":"kdd-2022-tutorial-on-large-scale-information-extraction-under-privacy-aware-constraints","status":"publish","type":"msr-project","link":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/project\/kdd-2022-tutorial-on-large-scale-information-extraction-under-privacy-aware-constraints\/","title":{"rendered":"KDD 2022 Tutorial on Large-Scale Information Extraction under Privacy-Aware Constraints"},"content":{"rendered":"
\n\t
\n\t\t
\n\t\t\t\t\t<\/div>\n\t\t\n\t\t
\n\t\t\t\n\t\t\t
\n\t\t\t\t\n\t\t\t\t
\n\t\t\t\t\t\n\t\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\n\n

Large-Scale Information Extraction under Privacy-Aware Constraints (opens in new tab)<\/span><\/a><\/h1>\n\n\n\n

<\/p>\n\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t<\/div>\n\t\t<\/div>\n\t<\/div>\n<\/section>\n\n\n\n\n\n

In this digital age, people spend a significant portion of their lives online and this has led to an explosion of personal data from users and their activities. Typically, this data is private and no-body else, except the user, is allowed to look at it. This poses interesting and complex challenges from scalable information extraction point of view: extracting information under privacy aware constraints where there is little data to learn from but need highly accurate models to run on large amount of data across different users. Anonymization of data is typically used to convert private data into pub-licly accessible data. But this may not always be feasible and may require complex differential privacy guarantees in order to be safe from any potential negative consequences. Other techniques involve building models on a small amount of seen (eyes-on) data and a large amount of unseen (eyes-off) data. In this tutorial, we use emails as representative private data to explain the concepts of scalable IE under privacy-aware constraints.<\/p>\n\n\n\n

Around 270 billion emails are sent and received per day and more than 60% of them are business to consumer (B2C) emails. At Microsoft, we have developed information extraction systems to extract relevant information from these emails for a large number of scenarios (e.g., flights, hotels, appointments, etc.), for thousands of sender domains (e.g., Amazon, Hilton, British Airways, etc.) and templates (HTML DOM structures)\u2014to power a number of AI applications (e.g., flight reminders, package tracking). As explained above, here are the challenges that we need to overcome to develop information extraction systems for emails:
Privacy<\/strong>: For legal and trust reasons, email and its derivatives should be accessible only to the person who it is intended to. Thus, we can\u2019t directly apply the web IE techniques used to extract information from webpages.
Efficiency<\/strong>: As we need to process billions of emails every day—different for different users—extraction models need to be very efficient.
Scalability<\/strong>: There are a large number of variations in the way information is presented in the emails. For example, a flight itinerary is represented in different ways by different providers.
Multi-lingual<\/strong>: Users are located across geographies, and hence, the information extraction systems need to work across multiple languages.<\/p>\n\n\n\n

To extract information from B2C emails, one needs to classify the emails, cluster them into possible templates, build models to extract information from them, and monitor the models to maintain a high precision and recall. How are the IE techniques for private eyes-off data different compared to that for eyes-on HTML data? How to get labeled data in a privacy preserving manner? What are the different techniques for generating semi-labeled data and learning from them? How to build scalable extraction models across a number of sender domains using different ways to represent the information? How to monitor these models with minimum human intervention? In this tutorial we address all these questions from various research to production perspectives.<\/p>\n\n\n\n\n\n\n\t\n\t\t\n\t\t\t\n\t\t\n\t\t
\"Rajeev<\/figure><\/td>\n\t<\/tr>\n\t
Rajeev Gupta<\/strong> is a Principal Applied Researcher at Microsoft Search Assistant & Intelligence (MSAI), India. He got his PhD from Indian Institute of Technology (IIT) Mumbai (Bombay) in Computer Science. He has more than 30 publications and 20 patents in the areas of data management, information extraction, and distributed computing in reputed conferences and journals such as TKDE, ICDE, VLDB, WWW, SIGMETRICS, CIKM, KDD, etc. He is currently working in applying AI for information extraction and mining enabling intelligence in Microsoft office for more than four years.<\/td>\n\t<\/tr>\n\t
\"Ranganath<\/figure><\/td>\n\t\t<\/tr>\n\t
Ranganath Kondapally<\/strong> is a is a Principal Applied Researcher at Microsoft Search Assistant & Intelligence (MSAI), India. He got his PhD in Computer Science from Dartmouth College in the area of computational complexity and streaming algorithms. His areas of interest include information extraction, machine learning algorithms, and complexity theory. He has numerous publications and patents in his name in the areas of information extraction, streaming algorithms, and virtual reality. Currently, he is working on information extraction and inferencing problems on bigdata, powering delightful personal assistant experiences.<\/td>\n\t<\/tr>\n<\/table>\n\n\n\n\n\n
  1. Difference between Information Extraction from web pages and that from emails
    1. IE from Semi-structured data<\/li>
    2. IE from Unstructured data<\/li><\/ol><\/li>
    3. Handling privacy issues \u00a0
      1. Anonymization of emails\u00a0while keeping it useful <\/em>for model\u00a0building\u00a0<\/li>
      2. Templatization <\/em>of emails <\/li><\/ol><\/li>
      3. Semi-supervised techniques to generate labeled-data\u00a0\u00a0
        1. Concepts of semi-supervision, using Structural similarity<\/li>
        2. Active learning <\/li>
        3. Transfer learning<\/li>
        4. Knowledge distillation, Teacher-student architecture<\/li>
        5. Weak labeling, data programming <\/li><\/ol><\/li>
        6. Text classification across languages with limited data
          1. Generating\u00a0labeled\u00a0data for English\u00a0<\/li>
          2. Using English data to create classifiers for other languages (Spanish, Portuguese, etc.)<\/li><\/ol><\/li>
          3. Handling\u00a0Scalability\u00a0issues in model building\u00a0\u00a0
            1. Adapting web IE techniques by writing wrappers– their limitations,\u00a0conjunctive\u00a0and disjunctive template (DOM) based models, etc.<\/li>
            2. Scalability issues due to number of scenarios, sender domains, and templates <\/li>
            3. Rule induction:\u00a0Programming\u00a0by examples <\/li>
            4. Machine learning approaches:\u00a0LR,\u00a0CRF,\u00a0LSTM, etc. <\/li>
            5. Combining rule induction and machine learning to get high precision and recall with high\u00a0coverage\u00a0
              1. Ensemble approach: Automated clustering, ML models to\u00a0identify\u00a0individual fields, generating\u00a0xpath<\/em>\u00a0based rules for each cluster.<\/li>
              2. Iterative approach: Use ML to create models which work well for seen templates and, to some extent, for\u00a0unseen templates; feed the data with probabilistic labels to rule-induction module; semi-supervised approach to remove discrepancies between ML and rule induction outputs; iterating multiple times to improve the performance and coverage.\u00a0<\/li><\/ol><\/li><\/ol><\/li>
              3. Efficient monitoring to\u00a0maintain\u00a0high precision and\u00a0recall\u00a0
                1. Sampling to\u00a0identify\u00a0precision and recall gaps <\/li>
                2. Anomaly detection algorithms<\/li><\/ol><\/li><\/ol>\n\n\n","protected":false},"excerpt":{"rendered":"

                  In this digital age, people spend a significant portion of their lives online and this has led to an explosion of personal data from users and their activities. Typically, this data is private and no-body else, except the user, is allowed to look at it. This poses interesting and complex challenges from scalable information extraction […]<\/p>\n","protected":false},"featured_media":0,"template":"","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","footnotes":""},"research-area":[13556,13555],"msr-locale":[268875],"msr-impact-theme":[],"msr-pillar":[],"class_list":["post-851687","msr-project","type-msr-project","status-publish","hentry","msr-research-area-artificial-intelligence","msr-research-area-search-information-retrieval","msr-locale-en_us","msr-archive-status-active"],"msr_project_start":"","related-publications":[],"related-downloads":[],"related-videos":[],"related-groups":[],"related-events":[],"related-opportunities":[],"related-posts":[],"related-articles":[],"tab-content":[],"slides":[],"related-researchers":[],"msr_research_lab":[],"msr_impact_theme":[],"_links":{"self":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/851687","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-project"}],"about":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/types\/msr-project"}],"version-history":[{"count":6,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/851687\/revisions"}],"predecessor-version":[{"id":952590,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-project\/851687\/revisions\/952590"}],"wp:attachment":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/media?parent=851687"}],"wp:term":[{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=851687"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=851687"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=851687"},{"taxonomy":"msr-pillar","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-pillar?post=851687"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}