The DataWeave Blog

Category: Company

Enterprise Data Security at DataWeave: Empowering Smarter Decisions with Seamless, Secure Data Management and Integration
At DataWeave, data security isn’t just about compliance—it’s about enabling peace of mind and better decision-making for our customers. Our customers rely on us not just for competitive and market intelligence but also for the seamless integration of critical data sources into their decision-making frameworks. To achieve this, we have built a security-first infrastructure that ensures organizations can confidently leverage both external and internal data without compromising privacy or protection.

Secure Data Integration: The Foundation of Smarter Decisions

Effective decision-making in today’s digital commerce landscape depends on combining multiple data sources—including first-party customer data, pricing intelligence, and business rules—into a unified framework. However, without the right security measures in place, businesses often struggle to operationalize this data effectively.

At DataWeave, we eliminate this challenge by offering:
- Integration with Leading Data Storage Solutions: Our platform seamlessly connects with data lakes and warehouses like AWS S3 and Snowflake, ensuring that businesses can easily ingest and analyze our data in real time.
- Support for Sandboxed Environments & Data Clean Rooms: Organizations can securely merge internal and external datasets without compromising confidentiality, unlocking deeper insights for pricing and business strategies.
- Automated Data Ingestion & Management: We simplify the process of integrating first-party data alongside competitive insights, allowing customers to focus on execution rather than infrastructure management.
Our Purpose-Built Security Framework

Handling millions of data points daily demands a security framework that is not only robust but also scalable and adaptable to evolving threats. DataWeave’s multi-tenant architecture ensures seamless data security without compromising operational efficiency.
- Multi-Tenant Architecture: Our system allows multiple customers to share the same application infrastructure while maintaining complete data isolation and security.
  - Tenants share infrastructure and computing resources but remain logically isolated.
  - Application-level controls ensure privacy while maximizing cost efficiency.
  - Centralized updates, maintenance, and easy scalability for new tenants.
- End-to-End Encryption & Access Controls: Every piece of data is encrypted both in transit and at rest. Role-based access controls (RBAC) restrict visibility to only authorized personnel, ensuring minimal risk of unauthorized data access.
Active Monitoring & Automated Compliance Management: We leverage automated access controls that adjust permissions dynamically as organizational roles evolve, ensuring that compliance is continuously maintained.

Certifications That Inspire Confidence

Data security is at the core of everything we do. Our compliance with the highest industry standards ensures that businesses can trust us with their sensitive data.

SOC 2 Type II Certification: DataWeave’s SOC 2 compliance is a testament to our commitment to stringent security protocols. This certification guarantees that we adhere to strict standards in data protection, availability, and confidentiality.

We implement a phased approach to security improvement:
- Prioritizing Critical Systems: To maximize impact, we prioritized systems that had the highest data security relevance and expanded the coverage thereafter. By addressing these priority areas, we were able to make meaningful security improvements early in the process.
- Automating Monitoring and Compliance: Partnering with Sprinto streamlined the compliance journey by automating key processes. This included real-time monitoring of our cloud environments, automated generation of audit-ready evidence, and integration with critical systems like AWS, Bitbucket, and Jira. These enhancements ensured efficient management of compliance requirements while reducing the burden on our teams.
- Fostering a Culture of Shared Responsibility: We conducted organization-wide training sessions to embed compliance as a shared responsibility across all teams. By educating employees on the importance of security practices and providing them with the tools to manage compliance autonomously, we established a security-first mindset throughout the company.
This systematic method allowed us to deliver immediate improvements while aligning long-term practices with industry’s best standards.

What This Means for Our Customers

By combining robust security with seamless data integration, DataWeave empowers businesses to:
- Optimize Price Management & Modelling: With secure access to real-time data, organizations can make informed pricing decisions that enhance profitability and market competitiveness.
- Run Advanced Simulations & Testing: Reliable, secure data enables businesses to model various pricing and assortment strategies before implementation, reducing risks and maximizing returns.
- Uncompromised Data Security: SOC 2 Type II compliance ensures stringent protocols to protect your data at every stage.
- Simplified Vendor Processes: Verified security certifications reduce friction during due diligence and onboarding, making it easier to partner with us.
- Aligned Standards: Our adherence to industry benchmarks reflects our commitment to meeting your expectations as a trusted technology partner.
- Scalable Operations: Expand across regions while maintaining full confidence in data privacy and security.
- Secure Collaboration: Share insights across teams with tools designed to protect sensitive information.
Our customers are increasingly looking to integrate their internal datasets with the external competitive intelligence provided by DataWeave. This can be a complex and risky process without the right security measures in place. We remove these roadblocks by providing a secure, scalable infrastructure designed to help businesses unify data without security concerns.

By ensuring seamless compatibility with key data storage platforms, such as Snowflake and AWS S3, we enable organizations to consolidate valuable first-party data with timely market insights. This integration empowers businesses to refine their pricing, assortment, and digital shelf strategies, thereby driving superior customer experiences—without the headaches of data security risks.

Security remains a top priority in everything we do. Our SOC 2 Type II-certified framework enforces rigorous encryption, access controls, and real-time compliance monitoring. We take on the burden of data security so our customers can focus on innovation and growth.

With DataWeave, businesses can confidently leverage secure data-driven decision-making to unlock new opportunities, optimize operations, and scale without compromise.

To learn more, write to us at contact@dataweave.com or request a consultation here.
January 21, 2025
How DataWeave Enhances Transparency in Competitive Pricing Intelligence for Retailers
Retailers heavily depend on pricing intelligence solutions to consistently achieve and uphold their desired competitive pricing positions in the market. The effectiveness of these solutions, however, hinges on the quality of the underlying data, along with the coverage of product matches across websites.

As a retailer, gaining complete confidence in your pricing intelligence system requires a focus on the trinity of data quality:
- Accuracy: Accurate product matching ensures that the right set of competitor product(s) are correctly grouped together along with yours. It ensures that decisions taken by pricing managers to drive competitive pricing and the desired price image are based on reliable apples-to-apples product comparisons.
- Freshness: Timely data is paramount in navigating the dynamic market landscape. Up-to-date SKU data from competitors enables retailers to promptly adjust pricing strategies in response to market shifts, competitor promotions, or changes in customer demand.
- Product matching coverage: Comprehensive product matching coverage ensures that products are thoroughly matched with similar or identical competitor products. This involves accurately matching variations in size, weight, color, and other attributes. A higher coverage ensures that retailers seize all available opportunities for price improvement at any given time, directly impacting revenues and margins.
However, the reality is that untimely data and incomplete product matches have been persistent challenges for pricing teams, compromising their pricing actions. Inaccurate or incomplete data can lead to suboptimal decisions, missed opportunities, and reduced competitiveness in the market.

What’s worse than poor-quality data? Poor-quality data masquerading as accurate data.

In many instances, retailers face a significant challenge in obtaining comprehensive visibility into crucial data quality parameters. If they suspect the data quality of their provider is not up to the mark, they are often compelled to manually request reports from their provider to investigate further. This lack of transparency not only hampers their pricing operations but also impedes the troubleshooting process and decision-making, slowing down crucial aspects of their business.

We’ve heard about this problem from dozens of our retail customers for a while. Now, we’ve solved it.

DataWeave’s Data Statistics and SKU Management Capability Enhances Data Transparency

DataWeave’s Data Statistics Dashboard, offered as part of our Pricing Intelligence solution, enables pricing teams to gain unparalleled visibility into their product matches, SKU data freshness, and accuracy.

It enables retailers to autonomously assess and manage SKU data quality and product matches independently—a crucial aspect of ensuring the best outcomes in the dynamic landscape of eCommerce.

Beyond providing transparency and visibility into data quality and product matches, the dashboard facilitates proactive data quality management. Users can flag incorrect matches and address various data quality issues, ensuring a proactive approach to maintaining the highest standards.

Retailers can benefit in several ways with this dashboard, as listed below.

View Product Match Rates Across Websites

The dashboard helps retailers track match rates to gauge their health. High product match rates signify that pricing teams can move forward in their pricing actions with confidence. Low match rates would be a cause for further investigation, to better understand the underlying challenges, perhaps within a specific category or competitor website.

Our dashboard presents both summary statistics on matches and data crawls as well as detailed snapshots and trend charts, providing users with a holistic and detailed perspective of their product matches.

Additionally, the dashboard provides category-wise snapshots of reference products and their matching counterparts across various retailers, allowing users to focus on areas with lower match rates, investigate underlying reasons, and develop strategies for speedy resolution.

Track Data Freshness Easily

The dashboard enables pricing teams to monitor the timeliness of pricing data and assess its recency. In the dynamic realm of eCommerce, having up-to-date data is essential for making impactful pricing decisions. The dashboard’s presentation of freshness rates ensures that pricing teams are armed with the latest product details and pricing information across competitors.

Within the dashboard, users can readily observe the count of products updated with the most recent pricing data. This feature provides insights into any temporary data capture failures that may have led to a decrease in data freshness. Armed with this information, users can adapt their pricing decisions accordingly, taking into consideration these temporary gaps in fresh data. This proactive approach ensures that pricing strategies remain agile and responsive to fluctuations in data quality.

Proactively Manage Product Matches

The dashboard provides users with proactive control over managing product matches within their current bundles via the ‘Data Management’ panel. This functionality empowers users to verify, add, flag, or delete product matches, offering a hands-on approach to refining the matching process. Despite the deployment of robust matching algorithms that achieve industry-leading match rates, occasional instances may arise where specific matches are overlooked or misclassified. In such cases, users play a pivotal role in fine-tuning the matching process to ensure accuracy.

The interface’s flexibility extends to accommodating product variants and enables users to manage product matches based on store location. Additionally, the platform facilitates bulk match uploads, streamlining the process for users to efficiently handle large volumes of matching data. This versatility ensures that users have the tools they need to navigate and customize the matching process according to the nuances of their specific product landscape.

Gain Unparalleled Visibility into your Data Quality

With DataWeave’s Pricing Intelligence, users gain the capability to delve deep into their product data, scrutinize match rates, assess data freshness, and independently manage their product matches. This approach is instrumental in fostering informed and effective decisions, optimizing inventory management, and securing a competitive edge in the dynamic world of online retail.

To learn more, reach out to us today!
January 22, 2024
AI-powered Product Matching: The Key to Competitive Pricing Intelligence in eCommerce
With thousands of products and hundreds of online retailers to choose from, the average modern-day shopper usually compares prices across several e-commerce sites effortlessly before often settling for the lowest priced option. As a result, retailers today are forced to execute millions of price changes per day in a never-ending race to be the lowest priced – without losing out on any potential margin.

Identifying, classifying, and matching products is the first step to comparing prices across websites. However, there is no standardization in the way products are represented across e-commerce websites, causing this process to be fairly complex.

Here’s an example:

What’s needed is a pricing intelligence solution that first matches products across several websites swiftly and accurately, and then enables automated tracking of competitor pricing data on an ongoing basis.

Pricing intelligence solutions already exist. What’s wrong with using them?

There are several challenges with the incumbent solutions in the market – the biggest one being that they don’t work in a timely manner. In essence, it’s like deferring the process of finding actionable information that helps retailers acquire a competitive advantage, and instead doing it in hindsight. Like an autopsy of sorts.

Here are the various solution types we have in the market today:
- Internally developed systems – Solutions developed by retailers themselves often rely on heavy manual data aggregation and have poor product matching capabilities. Since these solutions have been developed by professionals not attuned to building data crunching machines, they pose significant operational challenges in the form of maintenance, updates, etc.
- Web scraping solutions – These solutions have no data normalization or product matching capabilities, and lack the power to deliver relevant actionable insights. What’s more, it’s a struggle to scale them up to accommodate massive volumes of data during peak times such as promotional campaigns.
- DIY solutions – These solutions require manual research and entry of data. It goes without saying that due to the level of human intervention and effort required, they’re expensive, difficult to scale, slow, and of questionable accuracy.
As common as it is nowadays, AI has the answer

DataWeave’s competitive pricing intelligence solution is designed to help retailers achieve precisely the competitive advantage they need by providing them with accurate, timely, and actionable pricing insights enabled by matching products at scale. We provide retailers with access to detailed pricing information on millions of products across competitors, as frequently as they need it.

Our technology stack broadly consists of the following.

1. Data Aggregation

At DataWeave, we can aggregate data from diverse web sources across complex web environments – consistently and at a very high accuracy. Having been in the industry for close to a decade, we’re sitting on a lot of data that we can use to train our product matching platform.

Our datasets include data points from tens of millions of products and have been collected from numerous geographies and verticals in retail. The datasets contain hierarchically arranged information based on retail taxonomy. At the root level, there’s information such as category and subcategory, and at the top level, we have product details such as title, description, and other <attribute, value> relationships. Our machine learning architectures and semi-automated training data building systems, augmented by the skills of a strong QA team, help us annotate the necessary information and create labeled datasets using proprietary tools.

2. AI for Product Matching

Product matching at DataWeave is done via a unified platform that uses both text and image recognition capabilities to accurately identify similar SKUs across thousands of e-commerce stores and millions of products. We use an ensemble deep learning architectures tailored to NLP and Computer Vision problems specific to us and heuristics pertinent to the Retail domain. Products are also classified based on their features, and a normalization layer is designed based on various text/image-based attributes.

Our semantics layer, while technically an integral part of the product matching process, deserves particular mention due to its powerful capabilities.

The text data processing consists of internal, deep pre-trained word embeddings. We use state-of-the-art, customized word representation techniques such as ELMO, BERT, and Transformer to capture deeply contextualized text with improved accuracy. A self-attention/intra-attention mechanism learns the correlation between the word in question and a previous part of the description.

Image data processing starts with object detection to identify the region of interest of a given product (for example, the upper body of a fashion model displaying a shirt). We then leverage deep learning architectures such as VggNet, Inception-V3, and ResNet, which we have trained using millions of labeled images. Next, we apply multiple pre-processing techniques such as variable background removal, face removal, skin removal, and image quality enhancing and extract image signatures via deep learning and machine learning-based algorithms to uniquely identify products across billions of indexed products.

Finally, we efficiently distribute billions of images across multiple stores for fast access, and to facilitate searches at a massive scale (in a matter of milliseconds, without the slightest compromise on accuracy) using our image matching engine.

3. Human Intelligence in the Loop

In scenarios where the confidence scores of the machine-driven matches are low, we have a team of Quality Assurance (QA) specialists who verify the output.

This team does three things:
- Find out why the confidence score is low
- Confirm the right product matches
- Figure out a way to encode this knowledge into a rule and feed it back to the algorithm
In this way, we’ve built a self-improving feedback loop which, by its very nature, performs better over time. This system has accumulated knowledge over the 8 years of our operations, which is going to be hard for anyone to replicate. Essentially, this process enables us to match products at massive scale quickly and at very high levels of accuracy (usually over 95%).

4. Actionable Insights Via Data Visualization

Once the matching process is completed, the prices are aggregated at any frequency, enabling retailers to optimize their prices on an ongoing basis. Pricing insights are typically consumed via our SaaS-based web-portal, which consists of dashboards, reports, and visualizations.

Alternatively, we can integrate with internal analytics platforms through APIs or generate and deliver spreadsheet reports on a regular basis, depending on the preferences of our customers.

To summarize

The benefits of our solution are many. Detailed price improvement opportunity-related insights generated in a timely manner empower retailers to significantly enhance their competitive positioning across categories, product types, and brands, as well as ability to influence their price perception among consumers. These insights, when leveraged at a higher granularity over the long term, can help maximize revenue through price optimization at a large scale.

Our solution also helps drive process-based as well as operational optimizations for retailers. Such modifications help them better align themselves to effectively adopt a data-driven approach to pricing, in turn helping them achieve much smarter retail operations across the board.

All of this wouldn’t be possible if the product matching process, inherent to this system, was unreliable, expensive, or time-consuming.

If you would like to learn more about DataWeave’s proprietary product matching platform and the benefits it offers to eCommerce businesses and brands, talk to us now!
December 29, 2023
U.S. Prime Day Deals 2022: Promotion Intelligence First Look

As inflation hits another 40-year high at 9.1 percent, U.S. consumers geared up for their first sign of hope and relief in the form of anticipated discount buys – 2022 Amazon Prime Days, or so we thought. While Prime Days have grown to become a promotional period almost as important as Black Friday to digital shoppers, the combination of economic uncertainty, inflationary pressures, and supply chain challenges seemed to alter the discount strategy expected given activity seen during 2021 Prime Days.

Our analyst team has been hard at work aiming to provide a ‘first look’ at 2022 Prime Day Promotional Insights, tracking discounts offered across 46,000+ SKUs within key categories like Electronics, Clothing, Health & Beauty and Home, on seven major retailer websites – Amazon, Target, Best Buy, Sephora, Ulta, Lowe’s and Home Depot. Our analysis compares prices seen during Amazon Prime Day 2022 on July 12^th, to pre-Prime Day maximum value prices seen in the ten days leading up to Prime Days, to determine the average change in discounts offered during the promotional period. Below is a summary of our findings.

Competitive Promotions Give Amazon a Run for their Money

Amazon offered the greatest average discount enhancements for Electronics at 5.6 percent followed by Health & Beauty items at 5.1 percent, and Home products at 4.2 percent versus pre-Prime Day discounts seen across the categories considered within our analysis. The only category reviewed where average discounts were greater on a competitor’s website was on Target.com within the Clothing category. As seen below, Clothing on Target.com average discounts were 6.8 percent greater than pre-Prime Day offers, which was 2.6 percent higher than the average discounts offered for Clothing on Amazon.

Target Capitalizes on Growth Opportunity in Clothing Category

Diving deeper into the details of where Target won within the Clothing category, you can see a majority of their promotional activity took place within Women’s Accessories where discounts offered were 18.5 percent greater than those seen pre-Prime Day 2022, which was almost 15 percent greater than the discount enhancements seen on Amazon for Women’s Accessories. In fact, Women’s Shoes and Sneakers were the only two categories where the average discounts offered were greater on Amazon than on Target.com.

Overall, the discounts offered on Target.com within the Clothing category were primarily concentrated within items priced $40 and lower, but what was most interesting is that within the $10 and under price bucket, Target offered average discounts of over 11 percent whereas Amazon increased prices for these items on average by over 9 percent.

While most of the Clothing available on both Amazon and Target.com during Prime Days 2022 were offered without a price change, the greatest discount percentages tracked were within the range of 10-25 percent off on Amazon whereas Target chose to offer the bulk of their promotions at 25 percent off an up.

Strategic Promotional Strategies Defined at the Electronics Subcategory Level

When it comes to the Electronics category on Prime Day, the big question is always who will win the battle of the brands. Below shows the difference in average pricing and promotions discounts offered between products manufactured by Samsung versus Apple across each retailer platform, noting discounts were almost 3 percent greater on average for Apple versus Samsung products on Amazon, and Apple discounts were almost 5 percent greater on Amazon versus than those seen on Target.com.

Amazon wasn’t going all in on Apple however, as we saw ‘Alexa’ devices (Amazon products) available on Best Buy and Target websites also, but the discounts were almost 4 percent greater on Amazon versus Target and over 7 percent greater than the discounts seen on BestBuy.com.

While the average discounts offered within the Electronics category were greatest on Amazon (5.6 percent) versus Best Buy (3.9 percent) and Target (3.4 percent) as noted within the first chart of this blog and across brands and technologies considered above, the discounts offered on Amazon were strategically focused between 10-25 percent as seen below.

Amazon’s Electronics promotions were also targeted at smaller price points, items priced between $20-500, whereas Best Buy and Target offered greater promotions for electronics priced $500 and up than Amazon.

Below is a snapshot of price buckets tracked for Electronics available on BestBuy.com, highlighting where most of the promotional activity was targeted at products priced $50 and up during Prime Days 2022, with discounts ranging from 10 percent up to greater than 25 percent greater than pre-Prime day prices.

The standout categories were TVs on Target.com with discounts averaging nearly 12 percent greater than those seen pre-Prime day, and smartphones on BestBuy.com with discounts averaging just over 11 percent greater than those seen pre-Prime Day. The category with the greatest average discount enhancements seen on Amazon during Prime Days 2022 was for Wireless Headphones with an average discount of 8.7 percent.

Home is Where Amazon’s Heart Was on Prime Day

Amazon dominated offers within the Home categories, especially for products within mid ($40-100) and higher price ranges (items priced $200-500), with the bulk of the discounts offered between 10-25 percent. There was little to no promotional activity seen across all price points on Lowe’s or Home Depot’s websites within the categories we tracked, and most other competitive offers on Home products were seen on BestBuy.com for products priced from $50-500. Even a subcategory like Tools offered deeper average discounts on Amazon (4.7 percent) than discounts seen on HomeDepot.com (1.1 percent) or Lowes.com (0 percent).

For Large Appliances, Amazon was the only retailer to off any significant discount across each major subcategory with the greatest average discount being on Ovens at 6 percent, followed by Refrigerators at 4 percent. One caveat with this category, when we reviewed Large Appliance prices two weeks prior to Prime Days, we saw average price increases around 16.7 percent occurring on Amazon.

During Prime Days 2022 however, Amazon also offered top average discounts for small appliances, except for on Instant Pots which appeared to have greater average discounts on Target.com (5.9 percent versus 4.2 percent on Amazon), and Vacuum Cleaners which appeared to have the best promotion of appliances small and large at 13.8 percent average discount on BestBuy.com. Another subcategory deeply discounted on BestBuy.com was weighted blankets, which averaged discounts around 18.5 percent versus Amazon’s average discount at only 6.2 percent.

Health & Beauty Retailer Pricing Strategies Revealed

Given the importance Health & Beauty Brands placed on Prime Day sales last year, we had anticipated to see more offers, especially within pure-play beauty retail channels, than we did for this booming category.

Amazon drove most of the Health & Beauty offers seen averaging 5.1% discounts versus other retailers only offering less than 1% on average, but discounts were aimed at a targeted group of SKUs on Amazon, bringing the average discount lower overall. Most of the promotions offered on Amazon fell within mid-range price points ($20-50) and were discounted between 10-25 percent versus pre-Prime Day prices.

Target.com offered the most comparable discounts to Amazon for Health & Beauty products on average, but their strategy primarily focused on items within the $20 and lower price range with discounts ranging primarily between 10-25 percent.

More 2022 Prime Day Insights Coming Soon

We know the significance visibility to critical pricing and promotional insights play in enabling retailers and brands to offer the right discounts to stay competitive, especially during promotional periods like Prime Days. While this blog is intended to provide a ‘sneak peek’ into 2022 Prime Day insights for the U.S. market, we will be providing more extensive, global coverage and will proactively share new insights with the marketplace as they become available throughout the month of July.

Be sure to also check out our Press page for access to the latest media coverage on Prime Day insights and more. Don’t hesitate to reach out to our team if there is any particular category you are interested in seeing in more detail, or for access to more information on our Commerce Intelligence and Digital Shelf solutions.

July 14, 2022
Share of Keyword Search Cinco de Mayo 2022

As inflation continues to hike costs for consumers and supply chains challenge them to maintain loyalty, there is still an active audience willing to pay the ultimate price for the convenience of food and alcohol delivery. That being said, we analyzed 8 popular Retail and Delivery Intermediary websites and 11 popular ‘Cinco de Mayo’ keywords to see which Brands are predicted to win the battle of Digital Shelf Share of Search this holiday.

2022 Cinco de Mayo Share of Search Insights – Top Brands for ‘Cinco de Mayo’

Opportunities for Food & Bev on Cinco de Mayo

While most of our analysis focused on popular Cinco de Mayo food and beverage products, none of these brands populated on either Target (pictured on left below) or Walmart (pictured on right below) page 1 search results for the term ‘Cinco de Mayo’. Keyword search results for this term are dominated primarily by décor brands as indicated below.

Brands Achieving Top Share of Search for Food and Beverage Categories on Cinco de Mayo 2022

Share of Keyword Search Results – Alcohol Category

Three of the most popular alcohol types sought out during Cinco de Mayo are ‘Mexican Beer’, ‘Mezcal’, and ‘Tequila’. Below are the brands dominating Share of Keyword Search results on each of the major retail websites we researched.

AmazonFresh, Meijer, Kroger, and Sam’s Club Share of Search – Beer, Mezcal, and Tequila Keywords on Cinco de Mayo 2022

We also reviewed the same keyword performance across popular delivery intermediaries to see how Share of Keyword Search altered for ‘Mexican Beer’, ‘Mezcal’, and ‘Tequila’. The results are below for TotalWine, Instacart, Drizly and GoPuff:

TotalWine, Instacart, Drizly, and GoPuff of Search – Beer, Mezcal, and Tequila Keywords on Cinco de Mayo 2022

The keyword ‘Agave’ is also a popular search term within the alcohol category during the time leading up to Cinco de Mayo. We reviewed keyword search performance at various zip codes to see how price points that populated on page 1 search results varied given the change in median income. Below are the results:

Share of Search for Alcohol by Price Point and Zip Code on AmazonFresh

Share of Keyword Search Results – Grocery Categories

We also reviewed some of the most popular grocery items purchased during Cinco de Mayo by Keyword Share of Search results to see which brands are primed to win the Digital Shelf this year. Below are the results for Target.com and Walmart.com.

Walmart and Target Share of Search – Food and Beverage Keywords on Cinco de Mayo 2022

Below are the results for the same popular grocery items and alcohol keywords related to Cinco de Mayo and the page 1 results seen for Brand Share of Search on Safeway.com.

Safeway Share of Search – Food and Beverage Keywords on Cinco de Mayo 2022

Access to these types of real-time digital marketplace insights can enable retailers and brands to make strategic decisions and help drive profitable growth in an intensifying competitive environment. Be sure to reach out to our Retail Analytics experts for access to more details regarding the above analysis, and let us know what other holiday insights you’d be interested in seeing this year. Happy Cinco de Mayo!

May 4, 2022
2021 Cost-Push Inflationary Trends Ran Rampant, Impacting Holiday Discounts

Business has been anything but usual this holiday season, especially in the digital retail world. The holiday hustle and bustle historically seen in stores was once again occurring online, but not as anticipated given the current strength of consumer demand and the reemergence of COVID-19 limiting in-store traffic. While ‘Cyber Weekend’, Thanksgiving through Cyber Monday, continues to further its importance to retailers and brands, this year’s performance fell short of expectation due to product shortages and earlier promotions that pulled forward holiday demand.

Holiday promotions were seen beginning as early as October in order to compete with 2020 Prime Day sales, but discounting, pricing and availability took an opposite direction from usual. This shift influenced our team to get a jump start on our 2021 digital holiday analysis to assess how drastic the changes were versus 2020 activity, and to understand how much of this change has been influenced by inflationary pressures and product scarcity.

Scarcity Becomes a Reality

Our initial analysis started by reviewing year-over-year product availability and pricing changes from January through September 2021, leading up to the holiday season, as detailed in our 2021 Cyber Weekend Preliminary Insights blog. We reviewed popular holiday categories like apparel, electronics, and toys, to have a broad sense of notable trends seen consistently throughout various, applicable marketplaces. What we found was a consistent decline in product availability over the last six months compared to last year, alongside an increase in prices.

Although retailers significantly improved stock availability in November and early December 2021, even digital commerce giants like Amazon and Target were challenged to maintain consistent product availability on their website as seen below. While small in magnitude, there is also a declining trend occurring again closer toward the end of our analysis period, post Cyber Weekend, across all websites included in our analysis.

Source: Commerce Intelligence – Product Availability insights for Home & Garden, Jewelry & Watches, Clothing & Shoes, Bed N Bath, Lighting & Ceiling Fans categories

Greater Discounts, Higher Prices?

With inflation at a thirty-nine year high, retailers and manufacturers have realized they can command higher prices without impacting demand as consumers have shown their willingness to pay the price, especially when threatened by product scarcity. Our assessment is that while some products and categories have responded drastically, manufacturers’ suggested retail prices (MSRPs) have increased nearly seven percent on average from January to December 2021. MSRP adjustments are not taken lightly either, as this is an indication increased prices will be part of a longer-term shift in product strategy.

Source: Commerce Intelligence – Pricing Insights for Bed & Bath, Electronics, Furniture, Healthy & Beauty, and Fashion categories on Amazon.com & Target.com each month in 2021 comparing price increases from January 2021 base

Our 2021 pre-Cyber Weekend analysis reviewed MSRP changes for select categories (Bed & Bath, Electronics, Furniture, Healthy & Beauty, and Fashion) on Amazon and Target.com, and found around forty-eight percent of products on Amazon and thirty-five percent of products on Target.com have increased their MSRPs year-over-year, but kept pre-holiday discount percentages the same.

Looking more specifically as to what year-over-year changes occurred on Black Friday in 2021, we observed MSRPs increasing across the board for all categories at various magnitudes. This indicates why 2021 discounts appeared to be greater than or equivalent to 2020 for many categories, when in reality consumers paid a higher price than they would have in 2020 for the same items.

Source: Commerce Intelligence – MSRP Pricing Insights for Bed & Bath, Electronics, Furniture, Healthy & Beauty, and Fashion categories on Black Friday November 27th, 2021, versus average MSRP pricing for the same SKU count from November 20-26th 2021

On Amazon.com, categories like health & beauty have already increase MSRPs by a much greater percentage and magnitude versus Target.com leading up to and during Black Friday 2021, while other categories like furniture have increased MSRPs evenly on average across both retail websites. The below chart cites a few specific examples of year-over-year SKU-level MSRP, promotional price, and discount changes within found within the electronics, furniture, fashion, and health & beauty categories.

Source: Commerce Intelligence – MSRP Pricing Insights for Bed & Bath, Electronics, Furniture, Healthy & Beauty, and Fashion categories on Black Friday November 27th, 2021, versus average MSRP pricing for the same SKUs on Black Friday November 26th, 2020.

Fewer, but Deeper Discounts

From October through early November 2021, fewer products were discounted compared to this same period in 2020, and the few that were saw much deeper discounts apart from the home improvement category. The most extreme example we saw in discounts offered was within furniture where only three percent of SKUs were on discount in 2021 compared to twenty-six percent in 2020. Interestingly, the magnitude of discount was also higher pre-Cyber Weekend 2021 versus 2020, but this trend was not exclusive to furniture and was also seen within electronics, health & beauty, and home improvement.

Source: Commerce Intelligence – Pricing Insights for Bed & Bath, Electronics, Furniture, Healthy & Beauty, and Fashion categories on Amazon.com & Target.com Pre-Black Friday average selling price during November 20-26th 2021 versus average selling price from November 13-19th 2021 compared to Pre-Black Friday average selling price during November 19-25th 2020 versus average selling price from November 12-18th, 2020.

Within the furniture category, the subcategories offering the greatest number of SKUs with price decreases on Black Friday 2021 were rugs by a wide margin, followed by cabinets, bed and bath, and entertainment units, but the magnitude of discounts offered were all under twenty percent.

Source: Commerce Intelligence – Pricing Insights for Bed & Bath, Electronics, Furniture, Healthy & Beauty, and Fashion categories on Amazon.com and Target.com on Black Friday November 27th, 2021, versus average pricing for the same SKUs from Pre-Black Friday November 20-26th 2021 and Black Friday November 26th, 2020, versus average pricing for the same SKUs from Pre-Black Friday November 19th-25th 2020

Accounting for this phenomenon could have been retailers’ attempts to clear inventory for SKUs which hadn’t sold even during the period of severe supply chain shortages. With more products selling at higher prices this year, retailers were also able to use fewer SKUs with greater discounts to attract buyer in hopes of filling their digital baskets with more full-priced goods, helping to protect margins heading in to Cyber Weekend. Scarcity threats also encouraged consumers to buy early, even when not on promotion, to ensure they would have gifts in time for the holidays.

The same trends seen pre-Cyber Weekend 2021 were also seen on Black Friday with a year-over-year decrease in the percentage of SKUs offered on discount versus 2020, and steeper price reductions for the discounted products which can also be attributed to the increase in MSRPs.

Source: Commerce Intelligence – Pricing Insights for Bed & Bath, Electronics, Furniture, Healthy & Beauty, and Fashion categories on Amazon.com and Target.com on Black Friday November 27th, 2021, versus average pricing for the same SKUs from Pre-Black Friday November 20-26th 2021 and Black Friday November 26th, 2020, versus average pricing for the same SKUs from Pre-Black Friday November 19th-25th 2020

2021 Black Friday Price Increases?

We all know Black Friday is all about price reductions, discounts and deals and so it’s rare to see actual price increases, yet for Black Friday 2021, trends ran counter to this. We observed price increases across all categories for around thirteen to nineteen percent of SKUs, with an average price increase of around fifteen percent in 2021 versus an average of only two percent in 2020.

Source: Commerce Intelligence – Pricing Insights for Bed & Bath, Electronics, Furniture, Healthy & Beauty, and Fashion categories on Amazon.com and Target.com on Black Friday November 27th, 2021, versus pricing for the same SKUs from Pre-Black Friday November 20-26th 2021 and Black Friday November 26th, 2020, versus average pricing for the same SKUs from Pre-Black Friday November 19th-25th 2020

At an account level, we noticed a few interesting differences happening on Black Friday 2021 versus 2020 regarding category price changes. On Target.com, almost ninety percent of the bed and bath SKUs analyzed had a price change on Black Friday in 2021 versus 2020 with eighty-two percent presenting a higher price year-over-year versus only around seven percent showing a decrease, where on Amazon nearly forty-four percent of bed and bath SKUs showed an increase in price and around thirty-eight percent showed a decrease. Except for the health and beauty category on Target.com, more than half of the SKUs in each category saw a price increase on Black Friday versus a price decrease.

Source: Commerce Intelligence – Pricing Insights for Bed & Bath, Electronics, Furniture, Healthy & Beauty, and Fashion categories on Amazon.com and Target.com on Black Friday November 27th, 2021, versus average pricing for the same SKUs on Black Friday November 26th, 2020.

The magnitude of year-over-year price changes seen on Black Friday 2021 was significant across all categories, but the magnitude of price increases found on Amazon.com within the health and beauty category outpaced the rest by far. We reviewed three hundred and sixty-five SKUs on Amazon.com within the health & beauty category and saw almost eighty-three percent of them had a price change with around thirty-one percent decreasing prices and around fifty-two percent increasing prices. This means that within the health & beauty category on Amazon.com, more than fifty percent of the SKUs tracked were sold at a one hundred and seventy-six percent higher price on average during Black Friday 2021 versus 2020.

Source: Commerce Intelligence – Pricing Insights for Bed & Bath, Electronics, Furniture, Healthy & Beauty, and Fashion categories on Amazon.com and Target.com on Black Friday November 27th, 2021, versus average pricing for the same SKUs on Black Friday November 26th, 2020.

The subcategories offering the greatest number of SKUs with price increases on Black Friday 2021 were cameras, followed by men’s fragrances, laptops, and desktops & accessories, but the magnitude of discounts offered were all under ten percent.

Source: Commerce Intelligence – Pricing Insights for Bed & Bath, Electronics, Furniture, Healthy & Beauty, and Fashion categories on Amazon.com and Target.com on Black Friday November 27th, 2021, versus pricing for the same SKUs from Pre-Black Friday November 20-26th 2021 and Black Friday November 26th, 2020, versus average pricing for the same SKUs from Pre-Black Friday November 19th-25th 2020

The Aftermath Post-2021 Cyber Weekend

Extending this analysis beyond the holiday weekend, we analyzed price change activity from December third through the ninth across the top US retailers (chart below) and found that price decreases have been very minimal, comparatively speaking. Though there was a spike in number of price decreases from December 8^th to the 9^th, the percentage of SKUs with price decreases was still very low (less than three percent). We anticipate this trend will continue into 2022.

Source: Commerce Intelligence – Pricing insights for Home & Garden, Jewelry & Watches, Clothing & Shoes, Bed N Bath, Lighting & Ceiling Fans categories

A Sign of Things to Come

A confluence of inflationary trends, product shortages and consumer liquidity have driven many marketplace changes to occur simultaneously. Government programs in the form of stimulus checks, have put extra money in consumers’ hands, and so they’ve been more willing to spend. That, coupled with the shock in the supply chain, has motivated people to buy far ahead of the 2021 holiday season. Hence, retailers have needed to rely much less on across-the-board discounts. Promotions have been more strategic – we’ve seen deeper discounts over fewer products, likely used to draw consumers in to buy certain items, and once they’re there, customers are buying everything else at a non-discount level. When these factors once again normalize, we could see a return to the “race to the bottom” that has occurred since the financial crisis of 2008-2009, but for once, retailers may be able to maintain some pricing power as the 2021 holiday shopping season played out.

Even though performance was not as anticipated and holiday sales did not grow as rapidly as they did in 2020, Cyber Monday was still the greatest online shopping day in 2021. Through it all, retailers managed to keep their digital shelves stocked and orders filled in time for the holidays for the most part, running the risk of housing aged inventory if goods didn’t arrive in time. Despite predictions for steep promotions in January 2022, with supply chains still challenged and inflationary pressures still full steam ahead, we don’t anticipate much in the way of enhanced discounts to continue beyond the holidays.

Access to these types of real-time digital marketplace insights can enable retailers and brands to make strategic decisions like how and when to address inflationary pressures, while also supporting many other day-to-day operations and help drive profitable growth in an intensifying competitive environment. Continue to follow us in the coming weeks for a detailed 2021 year-end review across more retailers and categories. Be sure to reach out to our Retail Analytics experts for access to more details regarding the above analysis.

January 4, 2022
Are Your Digital Shelves Prepared for Green Monday?

Traditionally, retailers have staged multiple promotions between Black Friday and before Christmas Day to keep consumers excited about holiday shopping, so it’s easy to see why one more promotional day might fall into relative obscurity. As if ‘Early Start’ offers to Black Friday and extended ‘Cyber Weekend’ promotions weren’t enough to plan for, eBay added another day into the mix called ‘Green Monday’, much to the benefit of consumers, as it furthers the window of opportunity to secure a bargain during the holiday season.

Green Monday falls on the second Monday of December and has historically been one of the greatest sales days of the year for eBay, often attracting last-minute shoppers or those searching for last-minute deals. However, because of the 2021 Global Shipping Crisis, there is speculation that Green Monday may be the last chance this year to have items delivered in time for Christmas. For this reason, we believe it could turn into quite a fruitful event for participating retailers if it encourages procrastinating shoppers that traditionally spend closer to December 25^th to buy earlier in the season.

This isn’t the first year retailers outside of eBay have offered Green Monday promotions, however. Our team has been actively monitoring activity on this day from 2017 through present, to not only assess which retailers participate in the event, but also to understand how the discounts may change surrounding the event. The categories monitored include Apparel (Clothing, Shoes & Jewelry), Bed and Bath, and Home and Garden, and we’ve identified products offered on discount by comparing each applicable product’s price on Green Monday versus the most commonly seen price for the product offered throughout the month of December.

Better Promotions Than Boxing Day

Taking a closer look at 2020 Green Monday discounts within the categories and retailers analyzed, apart from Wayfair.com, we see all offered more SKUs on discount on Green Monday versus the days leading up to and out of the event. Kohls.com led the pack with around 93% of SKUs offered on discount, followed by Macys.com with 95%, and Wayfair.com with 83%. Overall, the number of SKUs on discount on Green Monday were greater than the SKUs offered on discount on Boxing Day, which is traditionally known as a great day to bargain shop.

Source: DataWeave Commerce Intelligence – Promotional Insights tracking Apparel, Bed & Bath, and Home & Garden category product’s online price on Green Monday 2020 in the US versus regular prices for the same products in the month of December each year.

What’s in Store for Green Monday 2021?

The insights we’ve tracked over the last four years have not indicated any signs to an end for Green Monday any time soon. As we see it, for consumers it is an extremely convenient time to order holiday gifts, and for retailers it is a good time to build brand trust and loyalty by fulfilling last minute orders at a great value, in time for the holidays.

Our prediction for the categories analyzed is to expect to see more retailers participate in Green Monday 2021 to a greater degree (more SKUs on sale and enhanced promotions). For retailers in this analysis, we would anticipate HomeDepot.com to enhance the number of offers to match 2020 competitive activity, and for Wayfair.com to look at increasing the number of offers on Green Monday versus the period leading into the event.

If you are interested in learning more about the details behind this analysis or our Promotional Insights solution, be sure to contact us. We can help you evaluate the effectiveness of your holiday promotional spend with access to near real-time marketplace insights on the brands, categories, and products your rivals promote, including discounts, campaign frequency and duration and more.

December 13, 2021
How Brands Can Outperform Rivals With Next-Gen Digital Shelf Analytics

As eCommerce grows in complexity, brands need new ways to grow sales and market share. Right now, brands face urgent market pressures like out-of-stocks, an influx of new competition and rising inflation, all of which erode profitability. As online marketplaces mature, more brands need to make daily changes to their digital marketing strategies in response to these market pressures, shifts in demand, and competitive trends.

eMarketer forecasts 2021 U.S. eCommerce will rise nearly 18% year-over-year (vs. 6.3% for brick-and-mortar), led by apparel and accessories, furniture, food and beverage, and health and personal care. The eCommerce industry is also undergoing fundamental changes with newer entities emerging and traditional business models evolving to adapt to the changed environment. For example, sales for delivery intermediaries such as Doordash, Instacart, Shipt, and Uber have gone from $8.8 billion in 2019 to an estimated $35.3 billion by the end of 2021. Similarly, many brands have established or are building out a Direct to Consumer (D2C) model so they can fully own and control their customer’s experiences.

In response, DataWeave has launched the next generation of our Digital Shelf Analytics suite to help brands across retail categories directly address today’s costly market risks to drive eCommerce growth and gain a competitive advantage.

“Our new enhancements help brands improve online search rank visibility and quantify the impact of digital investments – especially in time for the busy holiday season.”
~ Karthik Bettadapura, CEO and co-founder, DataWeave

The latest product enhancements provide brands access to tailored dashboard views that track KPI achievements and trigger actionable alerts to improve online search rank visibility, protect product availability and optimize share of search 24/7. Dataweave’s Digital Shelf Analytics platform works seamlessly across all forms of eCommerce platforms and models – marketplaces, D2C websites and delivery intermediaries.

Dashboard for Multiple Functions

While all brands share a common objective of increasing sales and market share, their internal teams are often challenged to communicate and collaborate, given differing needs for competitive and performance data across varying job functions. As a result, teams face pressure to quickly grasp market trends and identify what’s holding their brands back.

In response, DataWeave now offers executive-level and customized scorecard views, tailored to each user’s job function, with the ability to measure and assess marketplace changes across a growing list of online retail channels for metrics that matter most to each user. This enhancement enables data democratization and internal alignment to support goal achievement, such as boosting share of category and content effectiveness. The KPIs show aggregated trends, plus granular reasons that help to explain why and where brands can improve.

Brands gain versatile insights serving users from executives to analysts and brand and customer managers.

Prioritized, Actionable Insights

As brands digitize more of their eCommerce and digital marketing processes, they accumulate an abundance of data to analyze to uncover actionable insights. This deluge of data makes it a challenge for brands to know exactly where to begin, create a strategy and determine the right KPIs to set to measure goal accomplishment.

DataWeave’s Digital Shelf Analytics tool enables brands to effectively build a competitive online growth strategy. To boost online discoverability (Share of Search), brands can define their own product taxonomies across billions of data points aggregated across thousands of retailer websites. They can also create customized KPIs that track progress toward goal accomplishment, with the added capability of seeing recommended courses of action to take via email alerts when brands need to adjust their eCommerce plans for agility.

“Brands need an integrated view of how to improve their discoverability
and share of search by considering all touchpoints in the digital commerce ecosystem.”
~ Karthik Bettadapura, CEO and co-founder, DataWeave

Of vital importance, amid today’s global supply chain challenges, brands gain detailed analysis on product inventory and availability, as well as specific insights and alerts that prompt them to solve out-of-stocks faster, which Deloitte reports is a growing concern of consumers (75% are worried about out-of-stocks) this holiday season.

User and system generated alerts provide clarity to actionable steps to improving eCommerce effectiveness.

You also have visibility to store-level product availability, and are alerted to recurring out-of-stock experiences.

Scalable Insights – From Bird’s Eye to Granular Views

DataWeave’s Digital Shelf Analytics allows brands to achieve data accuracy at scale, including reliable insights from a top-down and bottom-up perspective. For example, you can see a granular view of one SKUs product content alongside availability, or you can monitor a group of SKUs, say your best selling ones, at a higher level view with the ability to drill down into more detail.

Brands can access flexible insights, ranging from strategic overviews to finer details explaining performance results.

Many brands struggle with an inability to scale from a hyper-local eCommerce strategy to a global strategy. Most tools available on the market solve for one or the other, addressing opportunities at either a store-level basis or top-down basis – but not both.

According to research by Boston Consulting Group and Google, advanced analytics and AI can drive more than 10% of sales growth for consumer packaged goods (CPG) companies, of which 5% comes directly from marketing. With DataWeave’s advanced analytics, AI and scalable insights, brands can set and follow global strategies while executing changes at a hyper-local level, using root-cause analysis to drill deeper into problems to find out why they are occurring.

As more brands embrace eCommerce and many retailers localize their online assortment strategies, the need for analytical flexibility and granular visibility to insights becomes increasingly important. Google reports that search terms “near me” and “where to buy” have increased by more than 200% among mobile users in the last few years, as consumers seek to buy online locally.

e-Retailers are now fine-tuning merchandising and promotional strategies at a hyper-local level based on differences seen in consumer’s localized search preferences, and DataWeave’s Digital Shelf Analytics solution provides brands visibility to retailer execution changes in near real-time.

Competitive Benchmarking

Brand leaders cannot make sound decisions without considering external factors in the competitive landscape, including rival brands’ pricing, promotion, content, availability, ratings and reviews, and retailer assortment. Dataweave’s Digital Shelf Analytics solution allows you to monitor share of search, search rankings and compare content (assessing attributes like number of images, presence of video, image resolution, etc.) across all competitors, which helps brands make more informed marketing decisions.

Brands are also provided visibility into competitive insights at a granular level, allowing them to make actionable changes to their strategies to stay ahead of competitors’ moves. A new module called ‘Sales and Share’ now enables brands to benchmark sales performance alongside rivals’ and measure market share changes over time to evaluate and improve competitive positioning.

Monitor competitive activity, spot emerging threats and immediately see how your performance compares to all rivals’, targeting ways to outmaneuver the competition.

Sales & Market Share Estimates Correlated with Digital Shelf KPIs

In a brick-and-mortar world, brands often use point of sale (POS) based measurement solutions from third party providers, such as Nielsen, to estimate market share. In the digital world, it is extremely difficult to get such estimates given the number of ways online orders are fulfilled by retailers and obtained by consumers. Dataweave’s Digital Shelf Analytics solution now provides sales and market share estimates via customer defined taxonomy, for large retailers like Amazon. Competitive sales and market share estimates can also be obtained at a SKU level so brands can easily benchmark their performance results.

Additionally, sales and market share data can also be correlated with digital shelf KPIs. This gives an easy way for brands to check the effect of changes made to attributes, such as content and/or product availability, and how the changes impact sales and market share. Similarly, brands can see how modified search efforts, both organic and sponsored, correspond to changes in sales and market share estimates.

Take Your Digital Shelf Growth to the Next Level

The importance of accessing flexible, actionable insights and responding in real-time is growing exponentially as online is poised to account for an increasing proportion of brands’ total sales. With 24/7 digital shelf accessibility among consumers comes 24/7 visibility and the responsibility for brands to address sales and digital marketing opportunities in real-time to attract and serve online shoppers around the clock.

Brands are turning to data analytics to address these new business opportunities, enhance customer satisfaction and loyalty, drive growth and gain a competitive advantage. Companies that adopt data-driven marketing strategies are six times more likely to be profitable year-over-year, and DataWeave is here to help your organization adopt these practices. To capitalize on the global online shopping boom, brands must invest in a digital shelf analytics solution now to effectively build their growth strategies and track measurable KPIs.

DataWeave’s next-gen Digital Shelf Analytics enhancements now further a brand’s ability to monitor, analyze, and determine systems that enable faster and smarter decision-making and sales performance optimization. The results delight consumers by helping them find products they’re searching for, which boosts brand trust.

Connect with us to learn how we can scale with your brand’s analytical needs. No project or region is too big or small, and we can start where you want and scale up to help you stay agile and competitive.

November 30, 2021
How an American QSR (Quick Service Restaurants) improved its Business ROI Food Apps

Traditionally, Quick Service Restaurants (QSRs) such as McDonald’s or Burger King, have been strategically operating on a brick and mortar model. However, according to some studies, an average QSR generates as much as 75% of its sales from online orders.

With the advent of delivery apps such as Uber Eats and Doordash, a significant portion of QSRs’ business has moved to these platforms. The war to top rank on one of these platforms is an even greater feat. With each brand competing for the top listing, it’s much less about the dollars you pay and much more about optimizing your investments.

The relationship between QSR chains and food delivery apps has its advantages and disadvantages. One of the critical grouses QSRs have against food apps is the incremental marketing spend required to participate on the platform and the inability to measure the impact of their investment. What makes matters worse is the limitation in metrics even available to measure the impact – neither the food apps provide them, nor does anyone else.

At DataWeave, we have made it our mission to enable QSRs to not only define measurable metrics to achieve a positive ROI for food app marketing investments, but we also equip QSRs with the tools to track their competitive performance at granular, zip code-based level so that localized strategies can be modified as needed. Below is an example of a 1000+ store chain QSR we partnered with to optimize a pre-existing investment made with a large food aggregator app. Within months of engagement with us, they were able to achieve a 3X increase in sales without adding any additional marketing dollars.

Below are the pain points we identified and solved together:

1. No Defined Metric

Problem – No leading metric to track marketing performance

One of the first issues we realized was that sales was not a good metric for tracking marketing performance as it’s a lagging metric and doesn’t capture the issues that help grow or suppress sales.

Most of the sales are driven by rank in the cuisine category and searches for branded keywords. But, the QSR chain had no way to track these ranks.

In fact, 70%+ sales go to the first five restaurants for the category and keyword

Comparing ranking on food delivery platforms across different categories and times

Solution – Establish ranking as a clear marketing metric

By aggregating data across different food app platforms comprehensively, i.e. across locations, at different times of the day, we established the ranking of the QSR chain in critical categories and for priority keywords, identifying where they under or over-performed relative to the competition. As we did this daily- this became a straightforward metric that helped establish the performance of their marketing campaign.

2. Geographical & Categorical Challenges

Problem: Identifying poor-performing stores and zip codes

We realized it was not a simple exercise to identify well performing stores on food apps since sales depend on many factors such as competition, population of the area, local cuisine preference, etc.

Solution: Zip Code Ranking and Attributes

We tracked the ranking of each store within each Zip Code for keywords and created a list of poor-performing stores. We also extracted attributes such as estimated time of arrival (ETAs), Delivery Fee, Ratings, Reviews, etc., for each of these poor performing stores, to identify the reasons for the poor ranking.

Analysing key metrics at a store level – identifying worst & best performing stores

E.g., We realized 356 of the stores were not populating on first page results, primarily because of poor ratings and High ETAs. After the focused initiative, 278 of these stores started showing on the first page and increased sales by 23%.

3. Sensitivity Analysis Deficiency

Problem: Not clear about the contribution of Rating, ETAs, Fees, etc. on the Ranking

The exact ranking algorithms of these food apps are not publicly shared – so the QSR chain wasn’t clear which variable of rating, ETAs, fees, ad spend, or availability contributed more or less to the overall ranking.

Solution: Sensitivity analysis for measuring contribution

Comprehensive data for multiple zip codes in various timestamps was analyzed to determine which variable contributes most significantly to the rankings and when. We also conducted A/B testing – simultaneously testing two different variables, such as reducing ETAs at one store and improving ad spend at another, calculating which led to greater rank and sales impact.

For example, we realized reducing publicized ETA’s (even by decreasing the delivery radius) contributed much more to improve the rankings than changes to ratings.

4. An Unknown Competitive Landscape

Problem: Tracking competitor performance

For example, we found the QSR chain performed well in key urban centers, but the competition was doing even better, but there wasn’t a good way to track and compare the performance of the competitors.

Solution:

We started tracking the QSR chain and the competition for each of the metrics and started comparing performance.

Analysing competitive performance on key metrics such as ETA, Availability etc

We quickly realized ranking started quickly improving as we gained a slight edge in each metric against the competitors. For example, 5 minutes less ETA adds to higher ranking.

In six months of this exercise with the QSR chain, we improved the average ranking from 24 to 11 for the QSR chain, getting them featured on the first page.

5. Blind Advertising Investment Opportunities

Problem:

The QSR chain was not clear on which banners (Popular near you, National Favorites, etc.) to choose to invest in, and had to depend on the recommendations of the food platforms entirely.

They weren’t even provided a clear view of which position made the banner visible and at what rank among those banners was their promo visible. They were at times the 7th promo in the 6th banner, which has almost zero probability of being discovered by the user – this happened despite paying heavily for the banners.

Solution:

We aggregated data for all banners populated within each zip code and found out the ranking and in which position the QSR chain was visible.

Identifying and analysing right banners for advertising spends

The QSR chain invested in 630 zip code-based banners with guaranteed visibility, but our assessment indicated the banners were only visible in 301 zip codes. After selecting suitable banners for promotions, we improved visibility to 533 zip codes within enhancing the budget.

We are now using the same strategy for refining discounts, offers, promotions, and coupons.

6. Lack of Campaign Performance Monitoring

Problem: Unsure of the long-term impact of marketing spend

In general, increasing marketing spend does give a temporary boost to sales, but the QSR chain’s question was, how can we measure the long-term impact i.e., ranking keywords and the targeted zip codes.

Solution:

We created a simple widget for every marketing campaign which showed the rank for the keywords for selected zip codes before the campaign, during the campaign, and post the campaign, clearly establishing the midterm impact of the campaign. This constant monitoring allowed the QSR to also quickly pivot on their strategy on account of national holidays etc, and act accordingly.

7. Non-Existent ROI Measurement

Problem: Establishing the impact of ranking on sales

Though the QSR chain could track sales that were coming via the food app channel, they had no way of knowing incremental organic volume driven by marketing efforts.

One missing variable here was how much of extra sales could be attributed to improvement of QSR ranking?

Solution:

By combining the sales data with aggregated insights over time, we established for the QSR chain how much increase in sales they could anticipate from an increase in ranking, also knowing which changed variables led to the percentage of change increase.

So, in essence, we were able to tell the QSR chain that for each store how much sales would increase by improving ETAs, rating, ad visibility, availability, etc., enabling precise ROI calculations for each intervention they make for their stores.

Increasing sales by 3x within six months was only the beginning, and the journey of driving marketing efficiency using competitive and channel data has only just begun.

DataWeave for QSRs

DataWeave has been working with global QSR chains, helping them drive their growth on aggregator platforms by enabling them to monitor their key metrics, diagnose improvement areas, recommend action, and measure interventions’ impact. DataWeave’s strategy eliminates the dependence on food apps for accurate data. We aggregate food app data and websites to help you with analysis and the justification of marketing spend and drive 10-15% growth.

DataWeave’s strategy eliminates the dependence on food apps for accurate data. We aggregate food app data and websites to help you with analysis and the justification of marketing spend and drive 10-15% growth.

If you want to know learn how your brand can leverage Dataweave’s data insights and improve sales, then click here to sign up for a demo

September 21, 2021
[INFOGRAPHIC] 2020: The Year the World Navigated Uncertainty Together

The start of 2020 brought with it the promise of global economic growth. Markets in the US were on a steady rise we also witnessed demand from brands and retailers in Europe and the Middle East. All seemed to be on track to make it a year of plenty.

Out of nowhere, the end of the first quarter saw the world coming to a grinding halt. The world was held hostage by a global pandemic and the force with which we were hit, was unprecedented.

From February to mid-May we saw things come to a sharp halt. We at DataWeave seized this intermittent downtime to bolster our product offerings.

On the flip side, when the world did start opening May onwards, we saw completely new categories take center stage digitally. With new habits and trends taking shape, the pandemic single-handedly caused exceptional growth in the Food and Grocery Delivery intermediaries. Predictably, the rest of the world followed. Our existing customers saw the competition rise steeply with everyone coming online. We invested substantially in our Digital Shelf Analytics solutions after noticing that e-commerce was seeing a boom. 2020 saw brands making their online presence the new norm. This meant that small, medium and large enterprises had to now divert their spending to analytics and e-commerce.

It is interesting to note that the rise in the food and grocery delivery segment gave brands another channel to focus on vis a vis their presence. Brands that were available on these sites focused on how they could optimize their sales on these channels, which proved to be the front runners during the height of the pandemic. While the challenges and opportunities for both these segments overlapped and seemed similar, our solutions helped measure and optimize brand performance across all online channels. Some of the in-demand solutions and analytics we saw our customers use were; share of search, content audit, assortment and availability, pricing and promotions, and ratings and reviews.

There were mixed emotions in the market, with regard to the best use of marketing spends. Human resource and client cutbacks happened across the board. At DataWeave however, we had the pleasure of onboarding 25 new clients including retailers and brands ranging from food and grocery delivery, home improvement from across multiple geographies.

Infographics

Throughout the year, the work never ceased at DataWeave. The team showed incredible resilience while working remotely, making sure our deliverables were being taken care of, at all times. Due to the e-commerce boom and immense pressure from existing and new entrants in the digital space, our clients saw a need to gather more insights. With the given uptick, we are happy to report that our stellar 95%+ accuracy record for in-depth insights at scale, was maintained through the course of all the work done.

Looking forward to the year 2021:

In the US, the adoption of e-commerce accelerated as traditional brick and mortar stores shut down and pivoted. To put things into perspective, e-commerce adoption grew only by 4.3% from 2014 to 2019. In just three months in 2020, e-commerce adoption grew at 4.3%! Add to that, with approved vaccines making their way slowly to the public, we do anticipate the travel sector to open up and we look forward to working with new clients.

Nike’s Chief Executive, John Donahoe recently said, ” We know that digital is the new normal. The consumer today is digitally grounded and simply will not revert back…the shift to online sales could be a permanent trend.” We could not agree more! With online sales here to stay, brand and retailers’ requirements to keep their competitive edge will only continue to grow. We at DataWeave, look forward to delivering the results they want in this new year, and for the years to come.

January 19, 2021
Market Intelligence Platform with Kenshoo

We’re thrilled to announce that we have teamed up with Kenshoo to offer an integrated marketing solution that combines DataWeave’s digital shelf analytics and commerce intelligence platform with Kenshoo’s ad automation platform. This in turn, provides better recommendations on promotions to retailers and consumer brands.

As e-commerce surges, consumer brands can now promote their products through retail-intelligent advertising. Product discoverability, content audit, and availability across large marketplaces can be critical to a brand’s success. Using DataWeave’s digital shelf solutions, Kenshoo now can offer marketers greater visibility into a brand’s performance.

Even large retailers and agencies can use our commerce intelligence platform to improve their price positioning, address category assortment gaps, and more.

Through this partnership, Kenshoo – a global leader in marketing technology, can help its significant base of consumer brands and retailers invest their marketing dollars intelligently and in a timely manner.

At DataWeave, we have constantly strived to bring in a holistic approach to help our customers optimize their online sales channels. This partnership furthers our resolve in this direction. As we collectively strive to adjust to a post-COVID-19 world, we are observing an acceleration towards digital commerce. This acceleration and change in consumer behavior is going to be a lasting change, creating significant growth opportunities for both DataWeave and Kenshoo.

With this partnership, we look forward to helping our customers make timely, intelligent, and data-driven decisions to grow their business.

July 22, 2020
Coronavirus Outbreak: Impact on E-Commerce Retailers and Consumer Brands

The Coronavirus, otherwise known as COVID-19, has made landfall on U.S. shores. At the time of writing this article, there are over 230 confirmed cases in the country and 12 deaths. The growing unease about the virus, which has quickly accumulated 95,000+ confirmed cases globally, has, among other things, adversely affected businesses and stock markets the world over.

In the wake of this outbreak, U.S. based retailers and brands would be prudent to brace themselves and plan ahead to minimize disruptions as much as possible.

Businesses and consumers in China, the global epicenter of the epidemic, have been dealing with these challenges over the last couple of months. It’s likely that some of the trends observed in China would be mimicked in the U.S. as well, something that domestic retailers and brands would do well to study and prepare for.

The Inadvertent E-commerce Wave

When the outbreak happened in China, it caused an uptick in e-commerce adoption as shoppers were reluctant to step out of their homes and instead, opted to shop for their goods online.

Reports indicate that Chinese online retailer JD.com’s online grocery sales grew 215% YoY over a 10-day period between late January and early February. Similarly, Carrefour’s vegetable deliveries grew by 600% YoY during the Lunar New Year period. Online sales of Dettol, a disinfectant produced by Reckitt Benckiser, rose 643% YoY between 10 February and 13 February on China’s Suning.com.

In Singapore, another region affected by the virus more recently than in China, Lazada’s grocery arm, RedMart, and Supermarket chain, NTUC FairPrice, both reported an unprecedented surge in demand, which tested their delivery capabilities to the limit.

This bump in online sales isn’t just restricted to grocery, but other categories as well. Jean-Paul Agon, CEO of L’Oréal, recently said that online sales of the brand’s beauty products increased in China in February.

Given such a consistent shift in shopping behavior across coronavirus-affected regions, it’s logical to expect that a similar trend would be followed in the U.S. – in fact, it might already be underway.

A recent survey by Coresight Research indicated that 27.5% of U.S. respondents are avoiding public areas at least to some extent, and 58% plan to if the outbreak worsens. Of those who have altered their routines, more than 40% say they are “avoiding or limiting visits to shopping centers/ malls” and more than 30% are avoiding stores in general. The survey also found consumers will likely begin to avoid restaurants, movie theaters, sporting events and other entertainment venues.

Therefore, it’s essential for U.S. retailers and brands to swiftly energize their e-commerce readiness and be fully prepared to cater to the circumstances-induced shift in shopping behavior, inclined toward online.

A Logistical Nightmare

The most obvious area of impact for retailers and brands is in their supply chain and order fulfilment operations.

A large portion of consumer product manufacturers rely to some extent on China, and the potential impact of the virus on supply chain processes is inescapable. Chinese factories have been operating at partial capacity, impacting supply chains globally. This has largely affected highly popular e-commerce categories like consumer electronics, fashion and furniture.

Shares in the U.S. of furniture e-commerce retailer, Wayfair, fell as much as 26% toward the end of February, according to a Bloomberg report. The is particularly revealing, as the online retailer reportedly relies on China for half of its merchandise.

Retailers struggling to cope with this stress in their supply chain systems would do well to warn their customers beforehand about delays in deliveries, like AliExpress has just done.

For categories like CPG, as consumers increasingly shop online, retailers that offer Buy Online Pick Up In Store (BOPIS), should expect a surge in its adoption, and reinforce their online infrastructure and in-store operations to cater to the rising demand.

In addition to disruptions in the supply chain, several other mission-critical areas are likely to get affected too.

Keeping Up With The Online Surge

As with any event of this magnitude, the business implications reach far and wide. The following are a few areas that we’ve identified as critical, based on our experience working with retailers and brands. Being aware of and focusing on these issues are likely to alleviate some of the issues faced by consumers today.

Fair pricing: There have been several reports of price gouging on e-commerce platforms. Examples include 2-ounce Purell bottles being sold for $400 and face masks for up to $20. While these prices have mostly been set by third party merchants, brands are likely to face the flak from consumers. A recent Bloomberg article reported that online retailers still rely partly on employees to manually monitor these items. This approach has obvious limitations, such as products quickly reappearing on the website after being de-listed. Brands and e-commerce platforms will need to explore automated ways of controlling their online pricing practices at large scale.

3P merchant and counterfeit management: Often, unauthorized third-party merchants selling an original manufacturer’s goods are the ones who unreasonably inflate prices. These merchants tend to test the markets on online marketplaces with their pricing, which adversely affects the brand image of the manufacturer. Further still, they sometimes list counterfeit or fake goods that make incorrect or extravagant claims. Brands will need to swiftly identify and de-list these merchants from online marketplaces.

Ensuring stock availability: During times like these, it’s a common sight to see empty aisles at supermarkets selling items like canned food, water, paper products and personal care products. Consumers will benefit from brands monitoring their stock availability at stores, which will help them better align their supply chain operations to the rapidly changing demand patterns across the U.S. map. This way, efforts can be more targeted at regions with severe shortages.

Content compliance: Helium 10, a technology provider for Amazon sellers, reported that since 26 February, 90% of searches on Amazon are coronavirus related, and searches for hand sanitizers spiked to 1.5 million searches in February compared to 90,000 in November. As a result, to arrest exploitative practices, some online marketplaces have announced policy guidelines on product content claiming health benefits. Words like ‘Coronavirus‘, ‘COVID-19‘, ‘Virus‘ and ‘epidemic’ are, in fact, prohibited. Amazon has already de-listed several merchants claiming fraudulent cures. Ebay has gone as far as to ban all new listings for face masks, hand sanitizers, and disinfecting wipes, due to regulatory restrictions. In this context, retailers and brands will benefit from deploying tracking mechanisms that quickly identify offenders.

The areas of business presented above are by no means a comprehensive list for retailers and brands to rely on during this time. Still, these are critical impact areas for them to address, even as huge efforts are made toward managing highly stressed supply chains.

DataWeave Offers Support

The coronavirus outbreak is likely to get worse before it gets better. As we enter unchartered territories, DataWeave is offering to contribute in small ways, pro bono, by leveraging our expert talent and competitive intelligence technology platform, to address some of the challenges faced by retailers and brands.

We’re announcing a limited-time, no-cost offer to detect and report on price gouging, the presence of unauthorized third-party merchants, as well as stock availability across U.S. ZIP-codes. This offer will be valid for 4-6 weeks (timeline will be flexible based on how the outbreak develops) and limited to monitoring the top 10 U.S. online marketplaces, as well as critical product categories such as medicinal and hygiene-related products, emergency food items, survival-related products, fuel, etc.

Reach out to us for further details.

March 6, 2020
[INFOGRAPHIC] 2019 at DataWeave: Blazing New Trails

As another year comes to a close, we look back at 2019 with fond memories and look forward to the exciting new prospects of 2020. Take a trip with us as we highlight some of DataWeave’s milestones of the last twelve months.

Over the course of the year, DataWeave’s success has gone hand in hand with the evolution of retail and e-commerce, reinforcing the relevance of our technology platform.

Our rapid growth in the North American market is a reflection of how intense competition in the region is triggering the need for accurate, timely, and actionable competitive and market insights, as well as other avenues for retailers and brands to gain a competitive edge.

Last year, we saw a resurgence of big-box (omnichannel) retailers as they adopted innovative approaches to play to their strengths (their offline stores). Offering buy online, pick up in store (BOPIS) or click-and-collect options, rolling out price match guarantee programs, and expanding their partnerships with delivery services like Instacart, enabled these retailers to leverage the best of both the online and offline worlds to compete with e-commerce firms.

Amazon continues to dominate e-commerce with a daunting 38% share in the US. Still, the partnerships between brands and Amazon are increasingly being tested. Nike and Ikea recently joined the likes of Swatch and Birkenstock to sever ties with the retail behemoth. This seemingly growing trend is largely due to counterfeits continuing to leak through the system.

Brands that used to de-prioritize their focus on their eCommerce channel (as it often was only a small portion of their revenues) have come to realize that consumers use large marketplaces like Amazon not just to shop for products but also to perform product research. As a result, how these brands are represented and sold online impact their offline sales. And with the onset of BOPIS and click-and-collect initiatives, brands can now analyze this correlation even at a hyperlocal (ZIP-code) level.

Large marketplaces, for their part, have started taking advantage of the increasingly brand-agnostic shopping behavior of consumers by launching ad-platforms for brands and manufacturers, enabling them to boost their visibility online.

Due to such sweeping transformations to the market landscape, brands and retailers are increasingly looking more toward intelligent tech-based solutions to help them gain a competitive edge.

In order to effectively serve the growing need for competitive and market insights, we’ve pushed our platform to its limits and beyond. It’s our constant endeavor to innovate and improve. This is evident with the launch of a host of new features on our product suite, especially Brand Analytics – designed to enable consumer brands to protect their brand equity and optimize e-commerce performance.

One of the key factors that enabled us to achieve all the milestones we did is the aggressive hiring of some of the most skilled talent in the tech industry. Our team grew by 44% in 2019, giving us additional confidence to raise the bar on our capabilities and offer 95% accuracy in our data and insights to our customers consistently.

We’re encouraged by the fact that we’ve more than doubled as a business, year-over-year, for the past several years, without depending solely on growing the team, but also by consolidating our technology stack, optimizing our processes, and scaling our products.

Here’s a sneak peek into our performance in 2019:

2020 Vision

The upcoming year promises to be an exciting one for the retail industry and the consumer brand space at large. We plan to be at the helm and increase our footprint all around. There’s a strong focus to expand our US team and consequently, the business. While we continue to strengthen our roots in India, we will look toward other mature markets like the UK, Germany and the Middle East as well.

On other fronts, we’re gathering steam on new partnership engagements – consulting firms, ad tech firms, marketing agencies and complementary technologies. We will also expand our foray into the travel and delivery services verticals.

With our diversifying portfolio, we haven’t lost sight of one of the most important aspects of any successful company – its employees. We will continue to keep our employees engaged, motivated, and satisfied by providing vertical and horizontal career growth opportunities, conducting personalized training programs, organizing hackathons, fostering cross-team collaboration and learning, and encouraging everyone to periodically blow off some steam at company retreats and the ferociously fought in-house sports tournaments.

Here’s to a stellar 2020 of empowered retailers and brands. We wish them well as they navigate the dense competitive landscape, knowing that they have an ally in their corner with DataWeave.

January 20, 2020
Flaunt Your Deep-Tech Prowess at Bootstrap Paradox Hackathon Hosted by Blume Ventures
When DataWeave was founded in 2011, we set out to democratize data by enabling businesses to leverage public Web data to solve mission-critical business problems. Eight years on, we have done just that, and grown to deliver AI-powered competitive intelligence and digital shelf analytics to several global retailers and brands, which include the likes of Adidas, QVC, Overstock, Sauder, Dorel, and more.

As the company has grown, so has our team, which is now 140+ members strong. We’re still constantly on the lookout for smart, open, and driven folks to join us and contribute to our success.

And so, we’re excited to partner with Skillenza and Blume Ventures to co-host the Bootstrap Paradox Hackathon, where we are eager to engage with the developer community and contribute in our own way back to the startup ecosystem.

The event will be conducted as an offline product building competition, with a duration of 24 hours on August 3-4, 2019 at the Microsoft India office in Bengaluru. It will provide a platform for developers and coders to interact with and solve challenges thrown up by DataWeave and other Blume portfolio companies, such as Dunzo, Unacademy, Milkbasket, Mechmocha, and Locus.

Taking up DataWeave’s challenge during this Hackathon will give you a sneak peek into what our team works on daily. It’s no surprise that we have “At DataWeave, it’s a Hackathon every day!” plastered on our walls. After all, it’s not just all about intense work, but also a lot of fun and frolic.

The problems that we deal with are as exciting as they are hard. Some of our key accomplishments in technology include:
- Matching products across e-commerce websites at massive scale and at high levels of accuracy and coverage
- Using Computer Vision to detect product attributes in fashion such as a color, sleeve length, collar type, etc. by analyzing catalog images
- Aggregating data from complex web environments, including mobile apps, and across 25+ international languages
One of our more recent innovations has been in optimizing e-commerce product discovery engines, which dramatically improves shopper experience and purchase conversion rates. During the Bootstrap Paradox Hackathon, coders will get a chance to build a similar engine, with guidance and assistance from DataWeave’s technology leaders.

Data sets containing product information like title, description, image URL, price, category etc. will be provided, and coders will need to clean up the data, extract information on relevant product attributes and features, and index them, in the process of building the product discovery engine.

For more details on the challenge, register here on the Skillenza platform.

As a sweetener, the event also promises everyone a chance to win over 10 lakhs in prize money.

Simply put, if you love code, this is the place to be this weekend. See you there!
August 1, 2019
Compete Profitably in Retail: Leveraging AI-Powered Competitive Intelligence at Massive Scale

AI is everywhere. Any retailer worth his salt knows that in today’s hyper-competitive environment, you can’t win just by fighting hard – you have to do it by fighting smart. The solution? Retailers are turning to AI in droves.

The problem is that many organizations regard AI as a black box of sorts – where you can throw all your data (the digital era’s blessing that feels like a curse) in at one end and have miraculously meaningful output appearing out the other. The reality of how AI works, however, is a lot more complex. It takes a lot of work to make AI work for you – and then to derive value out of it.

Image Source: https://xkcd.com/1838

Following the advent of the digital era, businesses across industries, particularly retail, were left grappling with massive amounts of internal data. To make things worse, this data was unstructured and siloed, making it difficult to process effectively. Yet, businesses learned to leverage simple analytics to extract relevant data and insights to affect smarter decisions.

But just as that happened, the e-commerce revolution stirred things up again. As businesses of all shapes, sizes, and types moved online, they suddenly became a whole lot more vulnerable to other players’ movements than they were just about a decade ago, when buyers rarely visited more than one store before they made a purchase. In other words, retailers are now operating in entire ecosystems – with consumers evaluating a number of retailers before making a purchase, and a disproportionate number of players vying for the same consumer mindshare and share of wallet.

Thus, external data from the web – the largest source of data known to man at present – is becoming critical to business’ ability to compete profitably in the market.

Competing profitably in the digital era: Can AI help?

As organizations across industries and geographies increasingly realized that their business decisions were affected by what’s happening around them (such as competitors’ pricing and merchandize decisions), they started shifting away from their excessive obsession with internal data, and began to look for ways to gather external data, integrate it with their internal data, and process it all in entirety to derive wholesome, meaningful insights.

Simply put, harnessing external data consistently and on a large scale is the only way for businesses to gain a sustainable competitive advantage in the retail market. And the only way to practically accomplish that is with the help of AI. Many global giants are already doing this – they’re analyzing loads of external data every minute to take smarter decisions.

That said, though, what you need to know is that all this data, while publicly available and therefore accessible, is massive, unstructured, noisy, scattered, dynamic, and incomplete. There’s no algorithm in the world that can start working on it overnight to churn out valuable insights. AI can only be effective if enormous amounts of training data is constantly fed back into it, coaxing it to get better and more astute each time. However, given the scarcity of readily available training datasets, limited and unreliable access to domain-specific data, and the inconsistent nature of the data itself, a majority of AI initiatives have ended up in a “garbage in, garbage out” loop that they can’t break out of.

What you need is the perfect storm

At DataWeave, we understand the challenge of blindly dealing with data at such a daunting scale. We get that what you need is a practical way to apply AI to the abundant web data out there and generate specific, relevant, and actionable insights that enable you to make the right decisions at the right time. That’s why we’ve developed a system that runs on a human-aided-machine-intelligence driven virtuous loop, ensuring better, sharper outcomes each time.

Our technology platform includes four modules:

1. Data aggregation: Here, we capture public web data at scale – whatever format, size, or shape it’s in – by deploying a variety of techniques.

2. AI-driven analytics: Since the gathered data is extremely raw, it’s cleaned, curated, and normalized to remove the noise and prepare it for the AI layer, which then analyzes the data and generates insights.

3. Human-supervised feedback: Though AI is getting smarter with time, we see that it’s still far from human cognitive capabilities – so we’ve introduced a human in the loop to validate the AI-generated insights, and use this as training data that gets fed back to the AI layer. Essentially, we use human intelligence to make AI smarter.

4. Data-driven decision-making: Once the data has been analyzed and the insights generated, they can either be used as it to drive decision-making, or then integrated with internal data for decision-making at a higher level.

With intelligent, data-backed decision-making capabilities, you can outperform your competitors

Understandably, pricing is one of the most popular applications of data analytics in retail. For instance, a leading, US-based online furniture retailer approached us with the mission-critical challenge of pricing products just right to maximize sell-through rates as well as gross margin in a cost-effective and sustainable manner. We matched about 2.5 million SKUs across 75 competitor websites using AI and captured pricing, discounts, and stock status data every day. As a result, we were able to affect an up to 30% average increase in the sales of the products tracked, and up to a 3x increase in their gross margin.

DataWeave’s powerful AI-driven platform is essentially an engine that can help you aggregate and process external data at scale and in near-real time to manage unavoidably high competition and margin pressures by enabling much sharper business decisions than before. The potential applications for the resulting insights are diverse – ranging from pricing, merchandize optimization, determination of customer perception, brand governance, and business performance analysis.

If you’d like to learn more about our unique approach to AI-driven competitive intelligence in retail, reach out to us for a demo today!

June 13, 2019
2018 at DataWeave: A Year of Prolific Success and Growth

As we enter 2019, in the backdrop of DataWeave’s unprecedented growth and success, we decided to take a breath and look back at some of the highlights of our progress over the last 12 months.

DataWeave’s growth through the year has been complemented and influenced by the evolution of the retail sector, reinforcing the relevance of our technology platform.

Amazon continued to dominate the online retail landscape, now commanding a staggering 49% of US e-commerce. At the same time, several large retailers have taken sure-footed strides toward establishing a stronger e-commerce presence, which places them head to head against the Seattle-based retail behemoth. As a result, competitive intelligence is no longer a “good-to-have” but is fundamental to the survival and growth of both traditional and new-age retailers, enabling them to devise smarter, data-driven competitive strategies.

Consumer brands are continuing to figure out the dynamics of selling on online marketplaces, which happens to give them valuable access to a vast base of shoppers while simultaneously restricting their ability to influence the brand experience. In their quest to sell more through the e-commerce channel, while trying to safeguard the brand experience and loyalty, consumer brands have turned increasingly toward e-commerce performance platforms to augment their decision-making process.

These trends have reinforced our confidence in our technology platform, which aggregates and analyzes data from the Web at massive scale to deliver actionable competitive insights, as we’re well poised to address the evolving challenges presented to retailers and brands today.

In 2019, there are no signs of slowing down for DataWeave.

We will continue to execute strongly in high-growth regions, and especially in the US, which has, in a span of two years, become the largest revenue generating region for DataWeave. We will also build a stronger footing in Europe, with specific focus on the UK market.

With time, our historical repository of data increases in volume and granularity, which enables us to better serve the maturing space of Alternative Data. We have already witnessed highly encouraging inbound interest over the last year, and we expect this interest to rise significantly moving forward.

With great success, comes the need for great people. In 2019, we will aggressively expand our team across functions, organization levels, and regions. As always, DataWeave is on the lookout for people who flourish in a competitive environment and can propel us to the next stage of growth.

Our technology platform never ceases to impress in its ability to aggregate and analyze billions of data points accurately each day. As our pipeline swells and we onboard bigger and more diverse customers, the platform will consistently be pushed to its limits, driving further innovations and improved efficiency.

Over the following 12 months, on the strength of all the lessons learnt and successes achieved in 2018, we look forward to another challenging year of empowering retailers and consumer brands to compete profitably in the new world order.

Watch this space for more on DataWeave through the year!

January 22, 2019
CEO Speak: Serving the US Market, Hiring the Right Talent, And More
Recently, Karthik Bettadapura, Co-founder & CEO at DataWeave, was interviewed by Vishal Krishna, Business Editor at YourStory, in the Bay Area, California. They discussed DataWeave’s focus on the US market, challenges that retailers face today, DataWeave’s technology platform and hiring practices, and more.

The following is a transcript of the interview.

(The transcript has been edited for clarity and brevity)

Vishal Krishna (VK): You left India to come and conquer America, why is that?

Karthik Bettadapura (KB) : Just a bit of history — we started in 2011 and product development and research was based in Bangalore, and still is. At the end of the first 5 years, we realized that we built great technology, but we were not able to scale beyond a certain point [in India]. If we had to build a growing business, we had to look at other markets as well.

VK: Quickly, can you tell me what DataWeave does?

KB: We provide Competitive Intelligence to retailers and customer brands. We work with some of the largest brands and retailers out there and we provide them with analyses to compete profitably.

VK: You said you had marque clients in India, yet you didn’t want to stay there because you wouldn’t have scaled beyond a particular point. Why is that?

KB :The ticket size in India is still on the lower side. If you must build a sustainable business, you need access to a much larger customer base and we found that in the US.

VK: Let’s start from the basics. What are a few things that a startup should decide to do when coming to America?

KB: A few things:
- A good understanding of the market
- Learn fast about the market
- Build a team here, or a have a team here already doing some work initially
- Consider how your team back in India will go about doing things in your absence
- The last one is about your own personal journey. I was so used to walking into an office and interacting with people. You come here, and you are all alone!
VK: It’s a lonely journey. Doors don’t open all that easily and you’ve got to hustle. Why?

KB: For people here, you are an unknown entity. Why should they be trusting someone who does not have enough customers here or has not raised money here? We had two US-based customers when we came in. It’s an uphill task to ensure that customers trust you.

VK: Who was the first customer you personally met here and why was that meeting so important?

KB: The first customer I met here was a large, big box retailer, and the meeting was primarily focused around why they should trust us — how can they know that we would survive and serve them, as well as how we are better than some of the other guys out there.

VK: Can you tell us what DataWeave does for US retailers?

KB: For retailers, we provide competitive intelligence, primarily around pricing optimization and assortment analytics. In the US, a lot of retailers are shutting shop and filing for bankruptcy.

VK: Yeah, we saw Sears go through something like that.

KB: The reasons fall broadly into 3 categories:
- They failed to compete profitably with a lot of these new age businesses.
- The new age retailers offer superior customer experience. They have figured out a better assortment/product strategy.
- The third one is ‘Price’ — price is such an important feature.
  What we do is help these retailers optimize their strategies around pricing, assortment and promotions, eventually enabling them to compete profitably.
VK: Typically, customers pay you on the outcome, pricing, license or subscription?

KB: It’s a subscription-based model. There is a one-time setup fee and an ongoing subscription fee.

VK: So you plug into their data management system?

KB: Yes, but we can also have our product sit independently. Sitting out of their internal systems is a benefit for us as we don’t have to get into the entire loop of integrations into their internal systems right from Day 1. We prove our product works and then we integrate with their systems.

VK: How do you integrate? Is the CIO your target?

KB: No, we don’t sell to the CIO world. We sell to analytics, pricing, and merchandising teams.

VK: Can pricing alone give retailers a competitive edge?

KB: Yes, pricing is a big lever that retailers use. For example, last holiday season’s sale, Amazon and Walmart made 120 million price changes in just 2–3 days.

VK: So they change the prices so dynamically to compete with each other. Is this price war coming to India?

KB: It is happening in India already.

VK: How much data can DataWeave’s infrastructure ingest?

KB: We are a global platform — we have customers across the globe, not just the US or India. So, on a daily basis, we process data on around 120 million products.

VK: Talk a little bit on R&D quickly. Do you have your marketing team in the US?

KB: We have marketing teams in the US and India.

VK: And the engineering team?

KB: The engineering team is in Bangalore.

VK: For people who want to work in your company, what kind of talent are you looking for?

KB: We look at 4 broad talent areas:
- One is in the world of data acquisition, which addresses issues like how data can be aggregated from thousands of websites and millions of pages on an ongoing basis, and how this data can be stored.
- The second area is on what kind of insights can be generated using this data. This could be done using text analytics, image analytics, and other technologies. This includes process optimization, in terms of building efficient and scalable systems.
- The third area is on how well the data can be represented if we have a customer who wants 60–70 million data points to be consumed on a weekly basis.
- And the last area is on data modeling — what kind of insights can we eventually give to the customer? And, when I say insights, I mean specific actions.
VK: You want people who can handle massive scale and for that they should be good at linear regression.

KB: We value people who write good code. We primarily work in Python, and we use a lot of optimization techniques in the middle of the stack to help us scale.

VK: Would you do something for supermarkets?

KB: Absolutely. The largest offline supermarket in India is our customer.

VK: So what can you do for supermarkets?

KB: Offline retailers across the world are facing something that’s called showrooming. This is when a shopper walks up to a store, looks at and feels a product, then searches online to see it’s available at a better price. So we have retailers who are wary of this phenomenon. We also have retailers who are wary of diminishing customer loyalty. So they have to constantly ensure that they are priced better in the market and are not losing customers because of [online] pricing.

VK: How powerful are your algorithms?

KB: There is a dedicated team that works on our algorithms. These fall into several buckets. One is pure data scale algorithms — how do you build systems which ensure that you are able to efficiently query them in real time and get the desired output. The second one is — how do you keep improving your machine learning algorithms. For example, computer vision algorithms, text analytics algorithm, etc. The third — how do you keep experimenting effectively.

VK: What role can an MBA degree holder play in DataWeave?

KB: We have people who hold MBA degrees and are working in customer success, delivery management, marketing, and sales.

VK: Do you spend time in training?

KB: You do have some lead time if you are a fresher, but if you are a lateral hire, its expected that you keep the ball rolling. They should be able to learn and learn fast — learning is more important than knowing. So, we give a lot of importance to people who can learn and pick up things quickly – about our product, handling customer objections, etc.

*

Watch the whole video here or check out DataWeave’s website to know more about how we use data engineering and artificial intelligence to enable retailers and brands to compete profitably in the age of eCommerce.
December 7, 2018
Evaluating the Influence of Learning Models

Natt Fry, a renowned thought leader in the world of retail and analytics, published recently an article expounding the value and potential of learning models influencing business decision-making across industries over the next few years.

He quotes a Wall Street Journal article (paywall) published by Steven A. Cohen and Matthew W. Granade who claim that, “while software ate the world the past 7 years, learning models will ‘eat the world’ in the next 7 years.”

The article defines a learning model as a “decision framework in which the logic is derived by algorithm from data. Once created, a model can learn from its successes and failures with speed and sophistication that humans usually cannot match.”

Narrowing this down to the world of retail, Natt states, “if we believe that learning models are the future, then retailers will need to rapidly transform from human-learning models to automated-learning models.”

This, of course, comes with several challenges, one of which is the scarcity of easily consumable data for supervised learning algorithms to get trained on. This scarcity often results in a garbage-in-garbage-out situation and limits the ability of AI systems to improve in accuracy over time, or to generate meaningful output on a consistent basis.

Enabling Retailers Become More Model-Driven
As a provider of Competitive Intelligence as a Service to retailers and consumer brands, DataWeave uses highly trained AI models to harness and analyze massive volumes of Web data consistently.

Far too often, we’ve seen traditional retailers rely disproportionately on internal data (such as POS data, inventory data, traffic data, etc.) to inform their decision-making process. This isn’t a surprise, as internal data is readily accessible and likely to be well structured.

However, if retailers can harness external data at scale (from the Web — the largest and richest source of information, ever), and use it to generate model-driven insights, they can achieve a uniquely holistic perspective to business decision-making. Also, due simply to the sheer vastness of Web data, it serves as a never-ending source of training data for existing models.

DataWeave’s AI-based model to leverage Web data

Web data is typically massive, noisy, unstructured, and constantly changing. Therefore, at DataWeave, we’ve designed a proprietary data aggregation platform that is capable of capturing millions of data points from complex Web and mobile app environments each day.

We then apply AI/ML techniques to process the data into a form that can be easily interpreted and acted on. The human-in-the-loop is an additional layer to this stack which ensures a minimum threshold of output accuracy. Simultaneously, this approach feeds information on human-driven decisions back to the algorithm, thereby rendering it more and more accurate with time.

Businesses derive the greatest value when external model-based competitive and market insights are blended with internal data and systems to generate optimized recommendations. For example, our retail customers combine competitor pricing insights provided by our platform with their internal sales and inventory data to develop algorithmic price optimization systems that maximize revenue and margin for millions of products.

This way, DataWeave enables retailers and consumer brands to utilize a unique model-based decision framework, something that will soon be fundamental (if not already) to business decision-making across industry verticals and global regions.

As AI-based technologies become more pervasive in retail, it’s only a matter of time before they’re considered merely table stakes. As summarized by Natt, “going forward, retailers will be valued on the completeness of the data they create and have access to.”

If you would like to learn more about how we use AI to empower retailers and consumer brands to compete profitably, check out our website!

Read Natt’s article in full below:

Steven A. Cohen and Matthew W. Granade published a very interesting article in the Wall Street Journal on August 19, 2018 — https://www.wsj.com/articles/models-will-run-the-world-1534716720

Their premise is that while software ate the world (Mark Andreessen essay in 2011, “Why Software is Eating the World”) the past 7 years, learning models will “eat the world” in the next 7 years.

A learning model is a decision framework in which the logic is derived by algorithm from data. Once created, a model can learn from its successes and failures with speed and sophistication that humans usually cannot match.

The authors believe a new, more powerful, business opportunity has evolved from software. It is where companies structure their business processes to put continuously learning models at their center.

Amazon, Alibaba, and Tencent are great examples of companies that widely use learning models to outperform their competitors.

The implications of a model-driven world are significant for retailers.

Incumbents can have an advantage in a model-driven world as they already have troves of data.

Going forward retailers will be valued on the completeness of the data they create and have access to.

Retailers currently rely on the experience and expertise of their people to make good decisions (what to buy, how much to buy, where to put it, etc.).

If we believe that learning models are the future then retailers will need to rapidly transform from human-learning models to automated-learning models, creating two significant challenges.

First, retailers have difficulty in finding and retaining top learning-model talent (data scientists).

Second, migrating from human-based learning models to machine-based learning models will create significant cultural and change management issues.

Overcoming these issues is possible, just as many retailers have overcome the issues presented by the digital age. The difference is, that while the digital age has developed over a 20 year period, the learning-model age will develop over the next 7 years. The effort and pace of change will need to be much greater.

October 11, 2018
[INFOGRAPHIC] 2017 at DataWeave: A Year in Retrospect

And that’s a wrap! Another exciting year done and dusted, in which DataWeave continued to execute strongly through accelerated revenue growth, new customer wins, and expansion to heretofore unchartered regions.

Through the year, we engaged with retailers and consumer brands of all types and sizes, and our belief that actionable competitive insights will increasingly play a defining role in driving profitable growth in retail was reinforced. Competition was stiff, and more times than not, we came out on top due to our ability to process huge data-sets, and the unmatched accuracy of our insights.

Encouragingly, the emerging vertical of Alternative Data gained greater maturity, as adoption of non-traditional data sources from the Web by Asset Managers picked up steam.

Our extensive focus on the North American market yielded impressive results, and we’ve only just scratched the surface.

Other regions and verticals continued to contribute significantly, helping us close out the year with record sale volumes.

As we wind ourselves up again for another marathon year in 2018, we look back at some of our achievements across the board, including customer impact, technology leadership, and team contribution and growth:

Moving into 2018, we have a lot to look forward to.

We’ll roll out a new and improved version of our SaaS-based data visualization platform, built with greater focus on actionability and customizability for our customers. Feedback from early beta tests have already been promising.

As our team size swells, we’ll be on the lookout for passionate problem solvers, who thrive in a hyper-competitive environment, to join us and contribute to the next stage of our growth journey.

Across verticals, we are well on our way to digging our heels into the North American market. 2018 will also see us gain a more solid footing in the Alternative Data space.

With eCommerce adoption showing no sign of slowing down, demand in retail for competitive intelligence solutions is set to soar, and our proprietary data aggregation and analysis platform is up to the challenge of catering to this growing need.

Stay tuned for more from DataWeave in 2018!

January 11, 2018
DataWeave Wins 2016 BI Software Awards From FinancesOnline

After a thorough assessment of our product FinancesOnline, a well-known software review platform and SaaS leads generation source, awarded DataWeave Retail Intelligence with two of their prestigious industry awards. According to FinancesOnline, our specialized competitive intelligence product is a rare tool that handles different languages with ease, and it allows businesses to improve the margin of their products and be more competitive.

Currently, DataWeave Retail Intelligence holds two of the platform’s prominent awards: the 2016 Great User Experience Award given to products which facilitate complex operations and allow users to navigate an easy and familiar interface; and the 2016 Expert’s Choice Award, confirming that DataWeave employs a variety of unique mechanisms to produce valuable competitors’ insights, compares and measures metrics that matter to every online store. Both awards were given for the platform’s business intelligence software reviews category.

According to their DataWeave review here the experts believe DataWeave genuinely focused on making businesses more competitive instead of simply listing data that may not be actionable by the company. They were particularly fond of the advanced identification of weak and strong points, actionable insights, and assortment intelligence, but mentioned as well the positive aspects of combining internal analytics with market data the way DataWeave does it. They praised our efforts to surpass traditional functionality gaps arising from language and location restrictions, and seem to firmly believe that out well-planned integrations make DataWeave usable for all type of analysis. Continuing with this tempo, FinancesOnline’s B2B professional foresee DataWeave performing successfully in many areas other than retail.

December 8, 2016