Managed Document Review

Job Description: Our client was representing a large financial institution who was obliged to respond to s.33 notice - Time Frame: 2 Days

 

Background

The Australian Securities and Investments Commission (ASIC) regulates Australian companies, financial markets, and financial services organisations and professionals who deal and advise in investments, superannuation, insurance, deposit taking and credit. ASIC accomplish this by monitoring compliance with the law and taking enforcement action where necessary.

ASIC have a range of compulsory information-gathering powers to provide them with documents and information, and to compel organisations and individuals to attend an examination to answer questions and/or provide reasonable assistance.

The Challenges

Our client was representing a large Australian financial institution (the Bank) who was obliged to respond to a s.33 notice issued under the ASIC Act (the Notice).

The Notice required the Bank to produce various documentation relating to its handling, during an eighteen‑month period, of several default events in relation to corporate loan facilities held by the borrower. There was only one week remaining to respond to the Notice and assessment of relevance to specific questions in the Notice was required as well as determination of privilege and privilege basis.

Our client was told by the Bank it had, by selecting a wide date range to be certain to capture all relevant materials, identified a data set of approximately 30GB of unprocessed data. The Bank initially estimated this set comprised “approximately 10,000 – 15,000 emails” requiring review. This number did not include attachments.

The lack of certainty regarding the exact number of documents made it difficult for our client to determine appropriate resourcing or even know whether the review could be completed in the limited time available.

The Solution(s)

1. Provide Proposal Options for Consideration

We were able to provide our client with various options for resourcing the first level review (FLR) depending on the various (and wide range of) possibilities of the exact number of documents within the data set. These proposals meant our client could be confident, regardless of the number of documents, we could deliver the documents to them in a timely manner.

2. Allocating and Training a Sizeable Review Team

Our team has in-depth experience in responding to regulatory notices from a number of bodies so was able to
commence the training of a team of over 50 reviewers on the particular subject matter of the transaction. The team was tested on the training and was able to develop a strong understanding of the subject matter in a short period of time.

This also meant the team was ready to commence FLR immediately upon the documents becoming available.

3. Flexible Working Arrangements

The Review Team was able to commence work on a Saturday and continue during a weekend to meet the client’s deadlines.

4. Prioritising Custodians

After processing, the data set was found to contain approximately 80,000 documents, comprising 38,000 emails and 42,000 attachments – from around 25 custodians (mailbox holders).

We were able to work with our client to proceed with the review prioritising documents based upon which particular custodians our client believed were more likely to be central to the investigation.

Three categories of Priority were created. This then permitted FLR, QA and release of those documents to the client for second level review (SLR) in that order, minimising downtime for the SLR team and enabling them to commence reviewing highly relevant documents more quickly.

Outcome

Infographic.jpg

By utilising our processing resources globally and extended weekend shifts for our global Review Team in India, the process from receipt of the unprocessed data through to release to our client of the coded data was completed in just 4 days. This was from receipt of data late on a Thursday to completion by Monday afternoon.

The MDR review team itself completed the assessment of all 80,000 documents for responsiveness to the Notice and relevant documents for privilege within just two days.

This gave our client sufficient time within which it could conduct its SLR and ultimately enabled the Bank to meet the tight deadline imposed upon it in the ASIC notice.

Conclusion

We were able to effectively manage our client’s review project despite an initially uncertain number of documents within an extremely tight timeframe.

Download the full case study

Contact the team

Job Description: Our client, a large energy company, was in the process of divesting part of their business..

 

The Challenge

Our client, a large energy company, was in the process of divesting part of their business and was required to review a large volume of company data for release to the newly separated entity. The entity would be provided with material in a ‘data room’. The data room would contain operationally essential information, as agreed to in the divestiture agreement. However, our client needed to ensure that all commercially sensitive and personally identifiable data was excluded from the data set before release.

The official divestiture date was due to take place three weeks after review commencement. Provision of the data by the due date was a central component of the contract and non-compliance would have resulted in significant financial penalties if the new parent company was not able to commence operations on the agreed date.

The client needed to:

  1. Identify their commercially sensitive data or personal customer and employee information that was to be excluded from the transfer of data to the new entity; and
  2. Ensure that the new entity was provided with all the information that would be crucial to continuing operations after the divestiture.

By engaging Law In Order, the client could be confident that their commercially sensitive data would be retained. Moreover, our review team could process the data and complete the review prior to the
separation date and ensure there was no unnecessary delay in the transition period.

This client sought advice from our expert review workflow team as to how to provide the new entity with the appropriate data prior to the separation date.

Solution

1. Process the documents

  • Our technical team uploaded the native data onto the Relativity platform for review.
  • Due to the original structure and format of the data, the data was unable to be automatically de-duplicated however, we worked with the client to develop a solution through propagation of review coding to identify all instances of documents to be removed from the source data.
  • We also worked alongside the client to develop effective and accurate keyword searches and apply them to the data to create batches of documents for the review team to examine, including ongoing fine-tuning of those searches as the data was reviewed and further trends identified.

2. Review by our Managed Review Team for commercially sensitive and private data to exclude:

  • Database design and setup to suit our client’s review outcomes.
  • Provided training to the review team on redacting commercially sensitive data and personal customer and staff information.

3. Identify documents proposed to be excluded from the data room

  • Establish identifiable risk categories for the client, including high risk for commercially sensitive documents and material containing personally identifiable information (PII).
  • We prepared a list of all instances of the documents to be excluded from the data room to be handed over to the new company. We also worked with the client’s IT team to create a workflow to separate out the identified material from that to be included in the database.

Our expert review team worked closely with our client to reduce their reviewable set of potentially relevant documents from over 2.5 million documents to 250,000 by combination of keyword searches and propagation of relevance coding.

We provided the client with a list of documents to be excluded from the entity’s company’s data room. The Law In Order review team worked diligently to identify commercially sensitive data to make sure it was not passed onto the separate entity. The data room for the new company was ready to be handed over by the client’s deadline.

The large volume of time and resources necessary to ensure accurate document review can put at risk both the quality and speed of your client’s outcomes, when working without the right support. Let our expert review team provide you with the resources you need for a tight turnaround.

If you are interested in understanding how our services might benefit you and your team, please do not hesitate to contact us for an obligation free discussion.

Download the full case study

Contact the team

Job Description: Our client, representing the trustee of a large trust, was required to disclose documents....

 

The Challenge

Our client, representing the trustee of a large trust, was required to disclose documents pertaining to a share value transaction dispute.

Following the application of keyword searches, approximately 37,000 documents had to be reviewed within a short period of time, with the court ordered disclosure date fast approaching. The client had minimal resources with only two lawyers on the matter. While the client had originally approached Law In Order only for eDiscovery services, our Consulting team suggested that the Managed Document Review team could assist to accurately review these documents in time for the disclosure deadline.

Solution

1. Set up the Review Platform
Our eDiscovery team processed the data and de-duplicated documents to create a smaller set. Then they uploaded the documents onto our Relativity review platform and applied keyword searches to further reduce the data set to assist the legal team.

2. Review the Relevant Documents on an Online Database
Using the disclosure categories required in the proceedings, we developed tags on the review platform for the Review Team to code against. The Managed Document Review team then underwent a Project Briefing and training on the matter from the Review Briefing Note prepared by the client, which gave them background on the matter as well as greater insight into the review categories. The Review Team was then able to review the material, coding for general relevance and privilege as well as responsiveness to the individual categories.

Outcome 

The Managed Document Review team completed First Pass Review of the material in two weeks, allowing the client time to complete their own Second Level Review to meet their disclosure deadline. Without our Managed Document Review services, the client most likely would have had to hire additional staff and train them within a short period of time to complete the task. The database we provided was searchable and made it simpler for the client to access the key documents for the case. The client stated that Law In Order “made light work of a pretty tricky discovery for us”.

Your Project

The large volume of time and resources necessary to ensure accurate document review can put at risk both the quality and speed of your client’s outcomes, when working without the right support. Let our expert review team provide you with the resources you need for a tight turnaround.

If you are interested in understanding how our services might benefit you and your team, please do not hesitate to contact us for an obligation free discussion.

Download the full case study

Contact the team

Job Description: Our client was representing a large infrastructure company in proceedings with their lead contractor

 

The Challenge

Our client was representing a large infrastructure company in proceedings with their lead contractor regarding disputed payments of variation and extension of time claims, among other issues. Like many construction projects, the volume of documents was significant. In addition, the ten disclosure categories were very complex, many turning on the circumstances of individual disputed events. Following application of keyword searches and other measures to reduce the overall document set, an initial set of 105,000 documents was identified for review.

The biggest obstacle to meeting the disclosure deadline was the review and identification of relevance to the disclosure categories and their subcategories (one category having over 35 subcategories). The client had initially proposed that the documents should be reviewed by each category, which would require the documents to be potentially reviewed ten times for each of the categories it was responsive to. We devised a workflow to create two groups of categories and thereby review the documents only a maximum of two times. The experience and training of the review team allowed them to consider half of the categories at one time while reviewing one document. This not only saved the client money but also allowed us to predict the project duration and resulting cost.

Solution

1. Workflow
We developed and implemented a plan to limit review of each document to a maximum of two times by allocating the categories to groups and the review operated as followed:

Perth_Image-(1).jpg

This enabled the Review Team to narrow their focus to each group of categories as was relevant and in turn, enabled faster and more accurate review of the documents and a speedier completion of the review.

2. Review
Using the disclosure categories required in the proceedings, we developed tags on the review platform for the Review Team to code against. The Managed Document Review team then underwent a Project Briefing and training on the matter from the Review Briefing Note prepared by the client, which gave them background on the matter as well as greater insight into the review categories. The Review Team was then able to review the material, coding for general relevance and privilege as well as responsiveness to the individual categories.

Outcome 

The workflow plan, tailor made for the client’s project, was vital to ensure all documents could be tagged accurately within the deadline. The customised review allowed the documents to be reviewed both diligently and efficiently. Due to thorough and extensive Review Briefings, the Review Team was able to gain understanding of the facts and circumstances of the matter and the review categories in a short period of time. Following this initial phase, the client engaged Law In Order to review additional tranches of documents in the next phases of their review project.

Your Project

The large volume of time and resources necessary to ensure accurate document review can put at risk both the quality and speed of your client’s outcomes, when working without the right support. Let our expert review team provide you with the resources you need for a tight turnaround.

If you are interested in understanding how our services might benefit you and your team, please do not hesitate to contact us for an obligation free discussion.

Download the full case study

Contact the team

 

Discovery & Review Services

Job Description: eDiscovery and Case Management hosting of 230 GB - Time Frame: 8 weeks

Our client required the extraction, culling and hosting of 1.2 million items for a fast turnaround request from a government regulator. A process strategy was developed to enable the firm to meet the deadlines. Execution included the electronic processing of all the evidence then presentation on a hosted case management platform for native file review. Once privilege and relevance of documents was determined Law In Order extracted and rendered the appropriate requisite documents in preparation for presentation to the regulator. The job was delivered on time & on budget.

Job Description: 778 GB of data to be reviewed - Cost Savings: $285,000 AUD

 

The Challenge

Our client—a large energy company—was involved in litigation over an environmental matter. Only one lawyer was assigned to the case, overseen by one independent consultant—an especially small team, given the 778GB of data collected for the case. In an effort to save costs, both sides agreed on a list of keywords to help identify relevant documents.

The Solution

De-duplication culled the client’s original count of 6.6 million documents down to 3 million documents. The list of keywords culled the document count significantly further, down to only 157,000 documents. However, with overly inclusive keywords like “environment” being used, it was evident there were still many non-responsive documents left in the collection to sort out—and that meant a lot of work remained for only one person to review.

Law In Order proposed using Relativity Assisted Review.

Justin Smith, Global Head of eDiscovery at Law In Order, oversees the data processing services and assists legal departments by offering technical solutions and workflows. He is a highly skilled legal technology consultant with over 12 years project management experience and a Certified Relativity Administrator. Justin has been involved in some of Australia’s most complex commercial litigations. He understands the complexities of unstructured data and offers solutions to make electronically stored information available for search, retrieval and review in a legally defensible manner.

“I am a great advocate of Assisted Review. We always put this option forward for any matter that has a high volume of documents,” said Justin. “For this matter, there were a number of reasons it was a great option for our client.”

Most notably, Law In Order calculated that if the one lawyer had to review all 157,000 documents linearly— strictly to identify relevant documents—the review would take 1,570 hours to complete. With the lawyer open to alternative processes, the decision was made. The team would use Technology Assisted Review.

Relativity_Diagram.jpg

The Technology Assisted Review Process

To begin the workflow, Our team used the overly inclusive keyword-responsive document set of 157,000 documents.

“We generally don’t use keywords in addition to Technology Assisted Review, but in this scenario, it actually fit the bill. It’s important to have a decent amount of both non-relevant and relevant documents to train the software—and after applying the keywords, it still looked like we were left with a good mix of relevant and non-relevant documents,” said Justin.

The lawyer completed three review rounds—reviewing a random sample of 1,000 documents during each round—to train the software on responsive versus non-responsive. For each round, six to seven percent of the documents were identified as responsive.

From there, the lawyer tested Relativity’s accuracy by conducting a QC round. For this fourth round, Justin and his team took a statistical sample of documents based on a 95 percent confidence level and a two percent margin of error. This resulted in a sample of 2,226 documents—and the results were consistent, as six to seven percent of the documents in the sample were coded as responsive.

In addition to the consistent results, the lawyer overturned only 174 of the 2,226 documents in the QC round. In other words, he disagreed with the software’s coding decisions only 7.8 percent of the time, further demonstrating the computer’s accuracy. Moreover, he made an interesting discovery while reviewing the overturns.

“As he double-checked the overturns to see what led the computer to make an incorrect decision, he actually ended up agreeing with the software the majority of the time,” said Justin. “He realized his initial decisions were wrong.”

With this realization and the already low overturn rate in mind, the lawyer felt the computer had achieved a consistent level of accuracy for this case—even after just one QC round—and decided to stop review.

Using the logic it learned from reviewing nearly 5,000 documents—three training rounds and one QC round— Relativity categorized the remaining 152,000 documents. In the end, 27,122 documents were marked responsive, 117,635 were non-responsive, and 300 remained uncategorised.

“Because of the limited resources, the legal team had very little time to complete this review. But, they were able to get good results quickly using Assisted Review,” said Justin.

The Outcome

Relativity Assisted Review saved our client a considerable amount of money. Without this, the client would have likely hired a junior lawyer—generally billing at $200 AUD/hour—to manually review all 157,000 keyword-responsive documents. Assuming the junior lawyer reviewed strictly for relevance at a rate of 100 documents per hour, it would have taken 1,570 hours to complete the first-pass review and would have cost the client $314,000 AUD.

However, by using Relativity Assisted Review, the client’s senior lawyer was able to complete the first-pass review at an estimated cost of $29,000—a savings of approximately $285,000 AUD for our client.

“This matter is a great example of the efficiency and cost savings realised by the use of Assisted Review,” said Justin. “Our client was very pleased with the results.”

Contact the team

Download the full case study

Job Description: Novice eDiscovery users required to assess 40GB Lotus Notes Files - Time Frame: 1 week

The Challenge

Our client with no eDiscovery experience had to assess 40GB worth of Lotus Notes e-mail files. They wanted a quick, low cost option to access the information on hand in order to evaluate the scope of the issues and determine their approach.

The Solution

We ran the files through Venio which gave lawyers access to the data within hours. As the client needed to ascertain how connected the e-mails were in relation to the matter – they wanted a clear picture of the parties involved, our Solutions Consultants ran keyword searches and showed the social network map that visually shows who this person sent emails to which proved to be a great tool to the legal team.

The Outcome

The use of Venio allowed for the legal team to complete scoping in 2 days and the Discovery completed within 7 days. The job was delivered earlier than expected and under budget.

Contact the team

Job Description: Regulator request for information, unknown data size & scope - Time Frame: 10 days

The Challenge

Our client was tasked to meet a regulator request for information within a 10 day period. The initial data size was not known, nor was the content or discovery scope. Ultimately we received 400GB of email and free files from the file server.

The Solution

We ran the raw data into Venio and within 24 hours the lawyers were able to look at the first 100 GB batch of information first thing in the morning. We then sat down with the partner and the senior associates of the matter and were able to refine Keywords/Date Range/Custodians elements of the data very very quickly – ultimately there were 150 custodians and 450 individual items.

The Outcome

Project was delivered within the time frame.

Contact the team

Job Description: Fast discovery deadline - Time Frame: 3 days

The Challenge

Our client had 1.2TB of data they needed to assess to determine the scope of discovery.

The Solution

We received 1.2TB of data on three hard drives.

The Outcome

The data was ingested and indexed into Venio FPR within 3 days, allowing the lawyers to view, search and analyse the documents to accurately determine the best course forward in the matter.

Contact the team

eTrials & eArbitrations

Job Description: Last minute request for eTrial - Time Frame: 48 hours

 

The Challenge

Law In Order received a call at 5:00 pm on a Friday with an urgent request for quotation for a court ordered eTrial involving the provision of evidence display solution to be managed by Counsel. There were two parties, 5 barristers and 5 lawyers. Law In Order had already worked with the plaintiff to provide an eCourtbook. The trial was schedule to commence on Monday.

The Solution

Law In Order responded with a quotation within 10 minutes of the request and received the go ahead from the client to proceed by 5:40 pm.

The Outcome

Law In Order worked through the weekend to ensure all the logistics were secured and set up the eTrial ready for proceedings to begin the following Monday at 10 am.

Relativity

Job Description: Rising business demand in 2017 meant larger and more complex jobs.

 

We had been relying on a legacy solution for processing our data into Relativity. When we took on an enormous job involving a company splitting into two entities, we knew our existing processing solution was not equipped to handle a project of this magnitude.

“Our team was manually mapping data fields and managing multiple imports and exports,” said Justin Smith, Global Head of eDiscovery at Law In Order. “Handling multiple data footprints was becoming tedious, and our storage costs to manage these systems were adding up.”

We had been considering Relativity Processing. This huge job — 8.5 TB of data with nearly 32 million documents — would be the perfect opportunity to put it to the test before full adoption.

DataFootprintIMage.png

 

Go Big or Go Home

Our client tasked us with analyzing its assets and identifying which documents belonged to the entity of the company that was sold off.

Phillip Buglass, Lead Consultant at Law In Order, strategised about the best way to get the data in front of review teams as quickly and accurately as possible.

“We were collecting and receiving continuous amounts of large data, so we couldn’t just hit and go,” Phillip explained. “With a broad collection of 32 million documents, we needed to cull junk data and only promote data relevant for review.”

Law In Order partnered with the Relativity Customer Success team to get the 8.5 TB of case data imported into Relativity Processing.

Then, devising a plan of attack to navigate the enormous data set, the eDiscovery team started an early case assessment (ECA) workflow to promote reviewable documents to reviewers. By doing so, the team parsed out specific file types within personal storage files—like emails, calendar items, and contacts—and applied de-duplication to reduce the amount of data from the get-go.

Law In Order’s prior solution involved loading data in phases, manually mapping fields, and continuously importing and exporting data, according to Justin.

“By the time our data was ready for review, we would have already quadrupled the data,” Phillip said. “With Relativity, we can just point and shoot, and the data is ready for attorneys to start reviewing immediately.”

Reduced by 75%

Using Relativity Processing and ECA reduced our overall data footprint by 75 percent with a 25 percent cost savings. In the second week of the project, Justin and Phillip had the following conversation over Skype:

Graphic.png

“The project’s results gave us the confidence we needed to adopt the entire Relativity platform for all our jobs,” Justin said. “We can continue to deliver top quality results to our clients in a fraction of the time.”

Contact the team

Download the full case study

 

Job Description: Our client received a tranche of disclosure from another party totaling around 55,000 documents.

 

The Challenge

Our client received a tranche of disclosure from another party totaling around 55,000 documents. Their aim was to review all 55,000 documents, but to prioritise the review to find documents similar in nature to their own key documents first.

The Solution

Relativity has various tools to identify similar documents.  Documents can be grouped by near textual duplicates, clustered by concepts or fed into a Technology Assisted Review (TAR) workflow.

In this matter where the client wanted to review all documents, but prioritise the order of review, the best solution was to deploy Relativity’s active learning workflow.

Active learning learns from decisions made on documents as they are coded in the review workflow. It uses these decisions to continuously deliver documents to a review queue based on what the software believes to be the next most similar document to those already coded as relevant.

As the review progresses, the review queue is continually updated based on the decisions being made.

In this matter we were able to take the documents that the client had already identified as key documents in their own disclosure and use these as a pre-coded set to train the system and kick start the active learning review queue.

The Outcome

The active learning project showed its value from the start.

Using only a very small seed set of around 260 key documents, the project returned a mean relevance rate of 45% for the first 2,000 documents. By the second 2,000 documents, the mean relevance rate had fallen to 24% and by the third set of 2,000 documents the mean rate had fallen to just 12%. In overall terms, the mean relevance rate fell from an initial peak of 63% (for the first 200 documents) to 6.8% by the end of the project, with the last 20,000 documents having a mean rate of just 1.54%.

The below graph illustrates how using active learning enabled the review team to review the most relevant documents sooner.

OverallMeanRelevance_Diagram.jpg

Contact our team

Download the full case study

 Can't find the resource you need?

Get in contact with the Law In Order team or call us on 1300 004 667.

Get in Touch