Category: Artificial Intelligence

  • Standard Reporting vs. Competitive Intelligence: What Retail Leaders Need to Know

    Standard Reporting vs. Competitive Intelligence: What Retail Leaders Need to Know

    Back in the day, pricing strategies were a lot easier. These days, not only do teams need to have robust standard price reporting workflows, but they also need to have the know-how and tools to gain and act on competitive intelligence. Retail leaders should prioritize automation and strategic thinking and ensure their teams have the tools, processes, and methodologies required to monitor the competition at scale and over the long term.

    Retail leaders who recognize the distinction between standard reporting and competitive intelligence are more likely to gain team buy-in, especially when developing pricing strategies that drive results. You can’t be everywhere at once, but you can optimize pricing strategies to stay ahead of the competition.

    This article has everything you need to know about the differences between standard reporting and competitive intelligence and how to use both to make your teams more effective than ever!

    Understanding the Distinction

    Standard price reporting is much like checking the weather to see if it’s stormy before grabbing a raincoat or sunhat. You need to do it to make essential, everyday choices, but it will not help you predict when the next storm is coming. Standard price reporting deals more with the short-term and immediate actions needed as opposed to long-term strategy.

    Don’t get us wrong, standard price reporting is still an essential responsibility of a pricing team’s function—but there’s more to it. It is also lower-tech than a competitive intelligence strategy and can rely on route heuristics.

    Think of it as data-in, data-out. It deals with pricing operations like:

    • Weekly price movements: Seeing which competitors, product categories, and individual items had pricing shifts in the short-term
    • Basic price indices: Outlining benchmarks to watch how your own, and your competitors’, products are trending in the market
    • Price competitiveness metrics: Setting thresholds that show whether your products are priced below, above, or equal to your competition for general trend reporting

    Standard price reporting is fundamental for operational teams that manage price adjustments in the short term. It can also help teams remain agile and reactive to market condition changes.

    It’s likely that your team already has standard reporting strategies or tools to help them with tactical execution. But are they harnessing competitive intelligence correctly with your help?

    Characteristics of Competitive Intelligence

    While standard price reporting is like checking the weather, competitive intelligence is like being a meteorologist who measures atmospheric changes, predicts storms, and scientifically analyzes weather patterns to keep everyone informed and in the know.

    Competitive intelligence goes well beyond simply tracking price movements and benchmarking them against a single set of standards. Competitive intelligence helps steer teams in a strategic direction based on insights from the market. It can drive long-term business success and is one of your best tools to ‘steer the ship’ as a retail leader.

    Here are some of the essential elements of competitive intelligence:

    • Strategic insights: Including but not limited to understanding your competitors’ pricing strategy, promotions, and product positioning
    • Market-wide patterns: Identifying trends based on geography, product category, or individual SKU across retailers to inform broader strategies
    • Long-term trends: Taking historical market and competitor data and combining it with real-time retail data to predict future price movements as shifts in consumer behavior to inform pricing strategies

    The pricing team serves as a critical strategic partner to senior leadership, delivering the cross-functional insights and market analysis needed to inform C-suite decision-making. By equipping executives with a holistic view of the competitive landscape, pricing gaps, and emerging trends, the team empowers leadership to align pricing strategies with broader business objectives.

    This partnership enables senior leaders to guide day-to-day pricing operations with confidence—ensuring tactical execution aligns with corporate goals, monitoring strategy effectiveness, and maintaining competitive agility. Through ongoing market intelligence and scenario modeling, the pricing function helps leadership proactively position the brand, capitalize on untapped opportunities, and future-proof revenue streams.

    Different Audiences, Different Needs

    As mentioned, there is a place for both standard price reporting and competitive intelligence. They have different roles to play, and different teams find them valuable. Since standard reporting mainly focuses on day-to-day shifts and being able to react to real-time changes, operational teams find it most useful.

    On the other hand, competitive intelligence is a tool that leadership can use to shape overarching pricing strategies. The insights from competitive intelligence drive operational activities over months and quarters, whereas standard reporting drives actions daily.

    To succeed in pricing, you need to rely on a combination of tactical standard reporting and competitive intelligence for long-term planning. With both, you can successfully navigate the ever-fluctuating retail market.

    Price Reporting for Operational Teams

    Your operational team is responsible for making pricing adjustments that directly impact sales volume. Automated data aggregation and AI-powered analytics can make this process faster and more accurate by eliminating the need for manual intervention.

    Instead of spending hours identifying changes, standard reporting tools surface the most critical areas that need attention and recommend adjustments. This helps operational teams react fast to shifting market conditions.

    Key functions of standard price reporting include:

    • Daily/weekly pricing decisions: Frequent price adjustments based on market trends will help your company remain competitive across entire product categories. With automated, real-time dashboards, your pricing team can monitor broad category-level pricing shifts and make necessary adjustments accordingly.
    • Individual SKU management: Not all pricing changes happen at the category level. Standard reporting also allows teams to view price and promotion changes on individual SKUs down to the zip code. It’s important to have targeted, granular insights when a change occurs even on a single SKU, especially because these individual changes are easy to miss. Advanced product matching algorithms can tie together exact products across retailers to monitor items conjointly. By incorporating similar product matching technologies beyond standard reporting, your teams can monitor individual price changes on comparable products.
    • Immediate action items: The best standard reporting tools alert pricing teams when there has been a change in competitor pricing and give them recommendations for what to change. If a competitor launches a flash sale or an aggressive discount program, your team should know as fast as possible which product to adjust. Without this functionality, teams can miss important changes or experience a delay in action that results in lost sales or customer perception.

    Competitive Intelligence for Leadership

    For Senior Retail Executives, Category Directors, and Pricing Strategy Leaders, pricing cannot only be about reacting to individual competitor price changes. Instead, you must proactively think about your market positioning and brand perception. Doing this without a complete competitive intelligence strategy can feel like throwing darts while blindfolded. Sometimes, you’ll hit the target, but mostly, you’ll miss or only come close. Competitive intelligence tools can help you hit that target every time. They leverage big data, artificial intelligence (AI), and predictive modeling to help you derive holistic insights to understand your current positioning relative to the current and future pricing landscape.

    Core strategic functions of competitive intelligence include:

    • Strategic planning: Competitive intelligence tools can help you forecast competitor behavior, economic shifts, and category-specific patterns you’d otherwise overlook (ex, price drops before new releases, subscription or bundling trends, or seasonable price cycles). Instead of reacting to a change, your team can already have made changes or at least know what playbook to implement.
    • Market positioning: Geographic pricing intelligence built into competitive intelligence tools can help you understand variations across locations and optimize multiple channels simultaneously. This can be the foundation of regional pricing strategies that factor in local economies and consumer perception.
    • Long-term decision-making: You can use competitive intelligence technology to align your pricing strategy with upcoming seasonal trends isolated using historical data, predicted economic shifts, and changes in customer purchasing behavior. This aggregate view of the pricing landscape will help you step out of the weeds and make better company decisions.

    From Data to Strategy – Transforming Basic Price Data

    Shifting your focus from isolated, reactive data to broader market trends is the key to going from basic price reporting to real competitive intelligence. Never forget the importance of real-time data, but know it’s your responsibility as a leader to bring a broader viewpoint to operations.

    Transforming from basic price data to competitive intelligence involves:

    1. Harnessing the data
      • Pattern recognition: Your solution should help you identify repeat pricing behaviors and competitor strategies
    2. Figuring out what to do with the data
      • Strategic implications: It should help you understand how your pricing changes will affect customer perception of your brand
    3. Doing something with the insights from your data
      • Action planning: The solution should help you create proactive strategies that position you as a market leader, leaving your competition to try to keep up with you instead of vice versa

    Leveraging Technology for Competitive Intelligence

    Technology is at the heart of leveling up your standard price reporting game. If you want industry-leading competitive intelligence, you can leverage DataWeave’s comprehensive pricing intelligence solution with built-in competitive intelligence capabilities and features for your operational teams.

    You can also uncover gaps and stay competitive in the dynamic world of eCommerce. It provides brands with the competitive intelligence they need to promptly adapt to market demand and competitors’ pricing. Stay ahead of market shifts by configuring your own alerts for price fluctuations on important SKUs, categories, or brands, all time-stamped and down to the zip.

    And since our platform relies on human-backed AI technology, you can have complete confidence in your data’s accuracy at any scale. If you want to bring a new strategic mindset to your pricing team, consider adding competitive intelligence to your tech stack. If you want to learn more, connect with our team at DataWeave today.

  • Portfolio Enhancement Through Price Relationship Management: Building Coherent Pricing Across Product Lines

    Portfolio Enhancement Through Price Relationship Management: Building Coherent Pricing Across Product Lines

    Do you remember when the movie Super Size Me came out? If you missed it, it was about the harmful effects of eating fast food too often. One aspect of the film that stands out is McDonald’s clever use of pricing to encourage consumers to buy bigger—and therefore more expensive—meals.

    Hungry patrons could upgrade their meal to a Super Size version for only a few cents more. In doing so, McDonald’s was able to capitalize on perceived value, i.e., getting more product for an apparent lower total price for the volume. It encouraged restaurant-goers to spend a little more while feeling like they got a great deal. It was a smart use of strategic pricing.

    There are hundreds of pricing relationship types like this one that pricing leaders need to be aware of and can use to their advantage when creating their team’s pricing strategy and workflows. You need to maintain profitable and logical price relationships across your entire product portfolio while keeping up with the competition. After all, the gimmick to Super Size would never have worked if the upgrade had been of less value than just buying another burger, for example.

    In this article, we’ll examine more real-world examples of pricing challenges so you can consider the best ways to manage complex price relationships. We’ll examine things like package sizes, brands, and product lines and how they’re intertwined in systematic price relationship management. Read on to learn how to prevent margin erosion, improve customer perception of your brand, and keep your pricing consistent and competitive.

    The Price Relationship Challenge

    Pricing is one of the most challenging aspects of managing a retail brand. This is especially true if you are dealing with a large assortment of products, including private label items, the same products of differing sizes, and hundreds, or even thousands, of competing products to link. Inconsistencies in your price relationship management can confuse customers, erode trust, and harm your bottom line.

    Let’s take a look at a few common pitfalls in portfolio pricing that you might run into in real life to better understand the impact on customer perception, trust, and sales.

    Pricing Relationship Challenges Retailers Need to Account For

    Private Label vs. Premium Product Pricing

    Let’s consider a nuanced scenario where price relationships between a retailer’s private label and premium branded products create an unexpected customer perception. Imagine you’re in a supermarket, comparing prices on peanut butter. You’ve always opted for the store’s private-label brand, “Best Choice,” because it’s typically the more affordable option. Here’s what you find:

    • Best Choice (Private Label) 16 oz – $3.50
    • Jif (National Brand) 16 oz – $3.25

    At first glance, this pricing feels off—shouldn’t the private label be the cheaper option? If a customer has been conditioned to expect savings with private-label products, seeing a national brand undercut that price could make them pause.
    This kind of pricing misalignment can erode trust in private-label value and even push customers toward the national brand. When price relationships don’t follow consumer expectations, they create friction in the shopping experience and may lead to lost sales for the retailer’s own brand.

    Value Size Relationships

    A strong value-size relationship ensures that customers receive logical pricing as they move between different sizes of the same product. When this relationship is misaligned, customers may feel confused or misled, which can lead to lost sales and eroded trust.

    Let’s look at a real-world example using a well-known branded product—salad dressing. Imagine you’re shopping for Hidden Valley Ranch (HVR) dressing and see the following pricing on the shelf:

    • HVR 16 oz – $3.99
    • HVR 24 oz – $6.49
    • HVR 36 oz – $8.99

    At first glance, you might assume that buying a larger size offers better value. However, a quick calculation shows that the price per ounce actually increases with size:

    • 16 oz = $0.25 per ounce
    • 24 oz = $0.27 per ounce
    • 36 oz = $0.25 per ounce

    Customers expecting a discount for buying in bulk may feel misled or frustrated when they realize the mid-size option (24 oz) is actually the most expensive per ounce. This mispricing could drive shoppers to purchase the smallest size instead of the intended larger, more profitable unit—or worse, lead them to a competitor with clearer pricing structures.

    Retailers must maintain logical price progression by ensuring that price per unit decreases as the product size increases. This not only improves customer trust but also encourages higher-volume purchases, driving profitability while maintaining a fair value perception.

    Price Link Relationships

    A well-structured price link relationship ensures customers can easily compare similar offerings of the same product and size. When the pricing across different versions or variations of the same item isn’t clear or consistent, it can confuse customers and damage trust, ultimately leading to missed sales and a negative brand perception.

    Let’s break this down with an example of a popular product—coffee. Imagine you’re shopping for a bag of Starbucks coffee and you see the following pricing on the shelf:

    • Starbucks Classic Coffee, 12 oz – $7.99
    • Starbucks Coffee, Mocha, 12 oz – $9.99
    • Starbucks Ground Coffee, Pumpkin Spice, 12 oz – $12.99

    At first glance, the product is the same size (12 oz) across all options, but the prices vary significantly. One might assume that the price difference is due to differences in quality or features, but what if there’s no clear indication of why the different flavors are priced higher than the standard?

    After investigating, you may realize that the only differences are related to different variants—like “Mocha” or “Pumpkin Spice” rather than any significant changes in the product’s core attributes. When customers realize they’re paying a premium for just different flavors, without any tangible difference in product quality, it can lead to feelings of confusion and frustration.

    Retailers must ensure that price links between similar offerings are justifiable by clearly communicating what differentiates each product. This avoids the perception that customers are being charged extra for little added value, building trust and encouraging repeat purchases. By maintaining transparent price link relationships, businesses can foster customer loyalty, increase sales, and drive better overall satisfaction.

    What is the Foundational Process to Tackle the Price Relationship Challenge?

    Now that we’ve reviewed several challenges brands face when pricing their products, what can be done about them?

    If you’re a pricing leader, you must create a robust pricing strategy that considers customer expectations, competitive data, sizing, and the overall value progressions of your product assortment. These are the three foundational steps to solve your price relationship challenges.

    1. First, you need to group products together accurately.
    2. Second, you need to establish price management rules around the group of related items.
    3. Third, you should set in place a process to review your assortment each week to see if anything is out of tolerance.

    This process is difficult, if not impossible, to manage manually. To effectively set up and execute these steps, you’ll need the help of an advanced pricing intelligence system.

    Implementation Strategy

    Want to know how to roll out a price relationship management strategy? Follow this implementation strategy for a practical way to get started.

    1. Set up price relationship rules: Determine which of your products go together, such as same products with different sizes or color options. Assign different product assortment groups and determine tolerances for scaling prices based on volume or unit counts.
    2. Monitoring and maintenance: Establish rules to alert the appropriate party when something is out of tolerance or a price change has been discovered with a competitive product.
    3. Exception management: Only spend time actioning the exceptions instead of pouring through clean data each week, looking for discrepancies. This will save your team time and help address the most significant opportunities first.
    4. Change management considerations: Think about the current processes you have in place. How will this affect the individuals on your team who have managed pricing operations? Establish a methodology for rolling this new strategy and technology out over select product assortments or brands one at a time to build trust with internal players.

    DataWeave offers features specifically built to help pricing teams manage pricing strategies. These applications can help you optimize profit margins and improve your overall market positioning for long-term success. A concerted effort to create brand hierarchies within your own product assortment from the get-go, followed by routine monitoring and real-time updates, can make all the difference in your pricing efforts.

    Within DataWeave, you can create price links between your products (value sizing) and those of the competition. These will alert you to exceptions when discrepancies are discovered outside your established tolerance levels. If a linked set of your products in different sizes shows inconsistent pricing based on scaled volumes, your team can quickly know how to make changes. If a competitor’s price drops significantly, you can react to that change before you lose sales.

    DataWeave even offers AI-driven similar product matching capabilities, which can help you manage pricing for private label products by finding and analyzing similar products across the market.

    If you want to learn more about price relationship management, connect with our team at DataWeave today.

  • Maximizing Competitive Match Rates: The Foundation of Effective Price Intelligence

    Maximizing Competitive Match Rates: The Foundation of Effective Price Intelligence

    Merchants make countless pricing decisions every day. Whether you’re a brand selling online, a traditional brick-and-mortar retailer, or another seller attempting to navigate the vast world of commerce, figuring out the most effective price intelligence strategy is essential. Having your plan in place will help you price your products in the sweet spot that enhances your price image and maximizes profits.

    For the best chance of success, your overall pricing strategy must include competitive intelligence.

    Many retailers focus their efforts on just collecting the data. But that’s only a portion of the puzzle. The real value lies in match accuracy and knowing exactly which competitor products to compare against. In this article, we will dive deeper into cutting-edge approaches that combine the traditional matching techniques you already leverage with AI to improve your match rates dramatically.

    If you’re a pricing director, category manager, commercial leader, or anyone else who deals with pricing intelligence, this article will help you understand why competitive match rates matter and how you can improve yours.

    Change your mindset from tactical to strategic and see the benefits in your bottom line.

    The Match Rate Challenge

    To the layman, tracking and comparing prices against the competition seems easy. Just match up two products and see which ones are the same! In reality, it’s much more challenging. There are thousands of products to discover, analyze, compare, and derive subjective comparisons from. Not only that, product catalogs across the market are constantly evolving and growing, so keeping up becomes a race of attrition with your competitors.

    Let’s put it into focus. Imagine you’re trying to price a 12-pack of Coca-Cola. This is a well-known product that, hypothetically, should be easy to identify across the web. However, every retailer uses their own description in their listing. Some examples include:

    How product names differ on websites - Amazon Example
    Why matching products is a challenge - Naming conventions on Target
    Match Rate Challenge - how product names differ on retailers - Wamlart
    • Retailer A lists it as “Coca-Cola 12 Fl. Oz 12 Pack”
    • Retailer B shows “Coca Cola Classic Soda Pop Fridge Pack, 12 Fl. Oz Cans, 12-Pack”
    • Retailer C has “Coca-Cola Soda – 12pk/12 fl oz Cans”

    While a human can easily deduce that these are the same product, the automated system you probably have in place right now is most likely struggling. It cannot tell the difference between the retailers’ unique naming conventions, including brand name, description, bundle, unit count, special characters, or sizing.

    This has real-world business impacts if your tools cannot accurately compare the price of a Coca-Cola 12-pack across the market.

    Why Match Rates Matter

    If your competitive match rates are poor, you aren’t seeing the whole picture and are either overcharging, undercharging, or reacting to market shifts too slowly.

    Overcharging can result in lost sales, while undercharging may result in out-of-stock due to spikes in demand you haven’t accounted for. Both are recipes to lose out on potential revenue, disappoint customers, and drive business to your competitors.

    What you need is a sophisticated matching capability that can handle the tracking of millions of competitive prices each week. It needs to be able to compare using hundreds of possible permutations, something that is impossible for pricing teams to do manually, especially at scale. With technology to make this connection, you aren’t missing out on essential competitive intelligence.

    The Business Impact

    Besides the bottom-line savings, accurately matching competitor products for pricing intelligence has other business impacts that can help your business. Adding technology to your workflow to improve match rates can help identify blind spots, improve decision quality, and improve operational efficiency.

    • Pricing Blind Spots
      • Missing competitor prices on key products
      • Inability to detect competitive threats
      • Delayed response to market changes
    • Decision Quality
      • Incomplete competitive coverage leads to suboptimal pricing
      • Risk of pricing decisions based on wrong product comparisons
    • Operational Efficiency
      • Manual verification costs
      • Time spent reconciling mismatched products
      • Resources needed to maintain price position

    Current Industry Challenges

    As mentioned, the #1 reason businesses like yours probably aren’t already finding the most accurate matches is that not all sites carry comparable product codes. If every listing had a consistent product code, it would be very easy to match that code to your code base. In fact, most retailers currently only achieve 60-70% match rates using their traditional methods.

    Different product naming conventions, constantly changing product catalogs, and regional product variations contribute to the industry challenges, not to mention the difficulty of finding brand equivalencies and private label comparisons across the competition. So, if you’re struggling, just know everyone else is as well. However, there is a significant opportunity to get ahead of your competition if you can improve your match rates with technology.

    The Matching Hierarchy

    • Direct Code Matching: There are a number of ways to start finding matches across the market. The base tier of the hierarchy of most accurate approaches is Direct Code matching. Most likely, your team already has a process in place that can compare UPC to UPC, for example. When no standard codes are listed, your team is left with a blind spot. This poses limitations in modern retail but is an essential first step to identifying the “low-hanging fruit” to start getting matches.
    • Non-Code-Based Matching: The next level of the hierarchy is implementing non-code-based matching strategies. This is when there are no UPCs, DPCIs, ASINs, or other known codes that make it easy to do one-to-one comparisons. These tools can analyze complex metrics like direct size comparisons, unique product descriptions, and features to find more accurate matches. They can look deep into the listing to extract data points beyond a code, even going as far as analyzing images and video content to help find matches. Advanced technologies for competitive matching can help pricing teams by adding different comparison metrics to their arsenal beyond code-based. 
    • Private Label Conversions: Up until this level of the hierarchy, comparisons relied on direct comparisons. Finding identical codes and features and naming similarities is excellent for figuring out one-to-one comparisons, but when there is no similar product to compare with for pricing intelligence, things get more complicated. This is the third tier of the matching hierarchy. It’s the ability to find similar product matches for ‘like’ products. This can be used for private label conversions and to create meaningful comparisons without direct matches.
    • Similar Size Mappings: This final rung on the matching hierarchy adds another layer of advanced calculations to the comparison capability. Often, retailers and merchants list a product with different sizing values. One may choose to bundle products, break apart packs to sell as single items or offer a special-sized product manufactured just for them. 
    Similar Size Mappings - product matching hierarchy - Walmart
    Similar Size Mappings - product matching hierarchy - Costco

    While at the end of the day, the actual product is the same, when there are unusual size permutations, it can be hard to identify the similarities. Technology can help with value size relationships, package variation handling, size equalization, and unit normalization.

    The AI Advantage

    AI is the natural solution for efficiently executing competitive product matching at scale. DataWeave offers solutions for pricing teams to help them reach over 95% product match accuracy. The tools leverage the most modern Natural Language Processing models for ingesting and analyzing product descriptions. Image recognition capabilities apply methods such as object detection, background removal, and image quality enhancement to focus on an individual product’s key features to improve match accuracy.

    Deep learning models have been trained on years of data to perform pattern recognition in product attributes and to learn from historical matches. All of these capabilities, and others, automate the attribute matching process, from code to image to feature description, to help pricing teams build the most accurate profile of products across the market for highly accurate pricing intelligence.

    Implementation Strategy

    We understand that moving away from manual product comparison methods can be challenging. Every organization is different, but some fundamental steps can be followed for success when leveling up your pricing teams’ workflow.

    1. First, conduct a baseline assessment. Figure out where you are on the Matching hierarchy. Are you still only doing direct code-based comparisons? Has your team branched out to compare other non-code-based identifiers?
    2. Next, establish clear match rate targets for yourself. If your current match rate is aligned with industry norms, strive to significantly improve it, aiming for a high alignment that supports maximizing the match rate. Break this down into achievable milestones across different stages of the implementation process.
    3. Work with your vendor on quality control processes. It may be worth running your current process in tandem to be able to calculate the improvements in real time. With a veteran technology provider like DataWeave, you can rely on the most cutting-edge technology combined with human-in-the-loop checks and balances and a team of knowledgeable support personnel. Additionally, for teams wanting direct control, DataWeave’s Approve/Disapprove Module lets your team review and validate match recommendations before they go live, maintaining full oversight of the matching process.
    4. The more data about your products it has, the better your match rates. DataWeave’s competitive intelligence tools also come with a built-in continuous improvement framework. Part of this is the human element that continually ensures high-quality matches, but another is the AI’s ‘learning’ capabilities. Every time the AI is exposed to a new scenario, it learns for the next time.
    5. The final step, ensure cross-functional alignment is achieved. Every one from the C-Suite down should be able to access the synthesized information useful for their role without complex data to sift through. Customized dashboards and reports can help with this process.

    Future-Proofing Match Rates

    The world of retail is constantly evolving. If you don’t keep up, you’re going to be left behind. There are emerging retail channels, like the TikTok shop, and new product identification methods to leverage, like image comparisons. As more products enter the market along with new retailers, figuring out how to scale needs to be taken into consideration. It’s impossible to keep up with manual processes. Instead, think about maximizing your match rates every week and not letting them degrade over time. A combination of scale, timely action, and highly accurate match rates will help you price your products the most competitively.

    Key Takeaways

    Match rates are the foundation of pricing intelligence. You can evaluate how advanced your match rate strategy is based on the matching hierarchy. If you’re still early in your journey, you’re likely still relying on code-to-code matches. However, using a mix of AI and traditional methods, you can achieve a 95% accuracy rate on product matching, leading to overall higher competitive match rates. As a result, with continuous improvement, you will stay ahead of the competition even as the goalposts change and new variables are introduced to the competitive landscape.

    Starting this process to add AI to your pricing strategy can be overwhelming. At DataWeave, we work with you to make the change easy. Talk to us today to know more.

  • From Raw Data to Retail Pricing Intelligence: Transforming Competitive Data into Strategic Assets

    From Raw Data to Retail Pricing Intelligence: Transforming Competitive Data into Strategic Assets

    Poor retail data is the bane of Chief Commercial Officers and VPs of Pricing. If you don’t have the correct inputs or enough of them in real time, you can’t make data-driven business decisions regarding pricing.

    Retail data isn’t limited to your product assortment. Price data from your competition is as important as understanding your brand hierarchies and value size progressions. However, the vast and expanding nature of e-commerce means new competitors are around every corner, creating more raw data for your teams.

    Think of competitive price data like crude oil. Crude or unrefined oil is an extremely valuable and sought-after commodity. But in its raw form, crude oil is relatively useless. Simply having it doesn’t benefit the owner. It must be transformed into refined oil before it can be used as fuel. This is the same for competitive data that hasn’t been transformed. Your competitive data needs to be refined into an accurate, consistent, and actionable form to power strategic insights.

    So, how can retailers transform vast amounts of competitive pricing data into actionable business intelligence? Read this article to find out.

    Poor Data Refinement vs. Good Refinement

    Let’s consider a new product launch as an example of poor price data refinement vs. good data refinement, which affects most sellers across industries.

    Retailer A

    Imagine you’re launching a limited-edition sneaker. Sneakerheads online have highly anticipated the launch, and you know your competitors are watching you closely as go-live looms.

    Now, imagine that your pricing data is outdated and unrefined when you go to price your new sneakers. You base your pricing assumptions on last year’s historical data and don’t have a way to account for real-time competitor movements. You price your new product the same as last year’s limited-edition sneaker.

    Your competitor, having learned from last year, anticipates your new product’s price and has a sale lined up to go live mid-launch that undercuts you. Your team discovers this a week later and reacts with a markdown on the new product, fearing demand will lessen without action.

    Customers who have already bought the much-anticipated sneakers feel like they’ve been overcharged now, and backlash on social media is swift. New buyers see the price reduction as proof that your sneakers aren’t popular, and demand decreases. This hurts your brand’s reputation, and the product launch is not deemed a success.

    Retailer B

    Imagine your company had refined competitive data to work with before launch. Your team can see trends in competitors’ promotional activity and can see that a line of sneakers at a major competitor is overdue for sale based on trends. Your team can anticipate that the competitor is planning to lower prices during your launch week in the hope of undercutting you.

    Instead of needing to react retroactively with a markdown, your team comes up with clever ways to bundle accessories with a ‘deal’ during launch week to create value beyond just the price. During launch week, your competitor’s sneakers look like the lesser option while your new sneakers look like the premium choice while still being a good value. Customer loyalty improves, and buzz on social media is positive.

    Here, we can see that refined data drives better decision-making and competitive advantage. It is the missing link in retail price intelligence and can set you ahead of the competition. However, turning raw competitive data into strategic insights is easier said than done. To achieve intelligence from truly refined competitive pricing data, pricing teams need to rely on technology.

    The Hidden Cost of Unrefined Data

    Technology is advancing rapidly, and more sellers are leveraging competitive pricing intelligence tools to make strategic pricing decisions. Retailers that continue to rely on old, manual pricing methods will soon be left behind.

    You might consider your competitive data process to be quite extensive. Perhaps you are successfully gathering vast data about your competitors. But simply having the raw data is just as ineffective as having access to crude oil and making no plan to refine it. Collection alone isn’t enough—you need to transform it into a usable state.

    Attempting to harmonize data using spreadsheets will waste time and give you only limited insights, which are often out of date by the time they’re discovered. Trying to crunch inflexible data will set your team up for failure and impact business decision quality.

    The Two Pillars of Data Refinement

    There are two foundational pillars in data refinement. Neither can truly be achieved manually, even with great effort.

    Competitive Matches

    There are always new sellers and new products being launched in the market. Competitive matching is the process of finding all these equivalent products across the web and tying them together with your products. It goes beyond matching UPCs to link identical products together. Instead, it involves matching products with similar features and characteristics, just as a shopper might decide to compare two similar products on the shelf. For instance private label brands are compared to legacy brands when consumers shop to discern value.

    A retailer using refined competitive matches can quickly and confidently adjust its prices during a promotional event, know where to increase prices in response to demand and availability and stay attractive to sensitive shoppers without undercutting margins.

    Internal Portfolio Matches

    Product matching is a combination of algorithmic and manual techniques that work to recognize and link identical products. This can even be done internally across your product portfolio. Retailers selling thousands or even hundreds of thousands of products know the challenge of consistently pricing items with varying levels of similarity or uniformity. If you must sell a 12oz bottle of shampoo for $3.00 based on its costs, then a 16oz bottle of the same product should not sell for $2.75, even if that aligns with the competition.

    Establishing a process for internal portfolio matching helps to eliminate inefficiencies caused by duplicated or misaligned product data. Instead of discovering discrepancies and having to fire-fight them one by one, an internal portfolio matching feature can help teams preempt this issue.

    Leveraging AI for Enhanced Match Rates

    As product SKUs proliferate and new sellers seem to enter the market at lightning speed, scaling is essential without hiring dozens more pricing experts. That’s where AI comes in. Not only can AI do the job of dozens of experts, but it also does it in a fraction of the time and at an improved match accuracy rate.

    DataWeave’s AI-powered pricing intelligence and price monitoring offerings help retailers uncover gaps and opportunities to stay competitive in the dynamic world of e-commerce. It can gather competitive data from across the market and accurately match competitor products with internal catalogs. It can also internally match your product portfolio, identifying product family trees and setting tolerances to avoid pricing mismatches. The AI synthesizes all this data and links products into a usable format. Teams can easily access reports and dashboards to get their questions answered without manually attempting to refine the data first.

    How AI helps convert raw data to pricing and assortment intelligence

    From Refinement to Business Value

    Refined competitive price data is your team’s foundation to execute these essential pricing functions: price management, price reporting, and competitive intelligence.

    Price Management

    Refined data is the core of accurate price management and product portfolio optimization. Imagine you’re an electronics seller offering a range of laptops and personal computing devices marketed toward college students. Without refined competitive data, you might fail to account for pricing differences based on regionality for similar products. Demand might be greater in one city than in another. By monitoring your competition, you can match your forecasted demand assumptions with competitor pricing trends to better manage your prices and even offer a greater assortment where there is more demand.

    Price Reporting

    Leadership is always looking for new and better market positioning opportunities. This often revolves around how products are priced, whether you’re making a profit, and where. To effectively communicate across departments and with leadership, pricing teams need a convenient way to report on pricing and make changes or updates as new ad hoc requests come through. Spending hours constructing a report on static data will feel like a waste when the C-Suite asks for it again next week but with current metrics. Refined, constantly updated price data nips this problem in the bud.

    Competitive Intelligence

    Unrefined data can’t be used to discover competitive intelligence accurately. You might miss a new player, fail to account for a new competitive product line, or be unable to extract insights quickly enough to be helpful. This can lead to missed opportunities and misinformed strategies. As a seller, your competitive intelligence should be able to fuel predictive scenario modeling. For example, you should be able to anticipate competitor price changes based on seasonal trends. Your outputs will be wrong without the correct inputs.

    Implementation Framework

    As a pricing leader, you can take these steps to begin evaluating your current process and improve your strategy.

    • Assess your current data quality: Determine whether your team is aggregating data across the entire competitive landscape. Ask yourself if all attributes, features, regionality, and other metrics are captured in a single usable format for your analysts to leverage.
    • Setting refinement objectives: If your competitive data isn’t refined, what are your objectives? Do you want to be able to match similar products or product families within your product portfolio?
    • Measuring success through KPIs: Establish a set of KPIs to keep you on track. Measure things like match rate accuracy, how quickly you can react to price changes, assortment overlaps, and price parity.
    • Building cross-functional alignment: Create dashboards and establish methods to build ad hoc reports for external departments. Start the conversation with data to build trust across teams and improve the business.

    What’s Next?

    The time is now to start evaluating your current data refinement process to improve your ability to capture and leverage competitive intelligence. Work with a specialized partner like DataWeave to refine your competitive pricing data using AI and dedicated human-in-the-loop support.

    Want help getting started refining your data fast? Talk to us to get a demo today!

  • How AI Can Drive Superior Data Quality and Coverage in Competitive Insights for Retailers and Brands

    How AI Can Drive Superior Data Quality and Coverage in Competitive Insights for Retailers and Brands

    Managing the endlessly growing competitive data from across your eCommerce landscape can feel like pushing a boulder uphill. The sheer volume can be overwhelming, and ensuring that data meets standards of high accuracy and quality, and the insights are actionable is a constant challenge.

    This article explores the challenges eCommerce companies face in having sustained access to high-quality competitive data and how AI-driven solutions like DataWeave empower brands and retailers with reliable, comprehensive, and timely market intelligence.

    The Data Quality Challenge for Retailers and Brands

    Brands and retailers make innumerable daily business decisions relying on accurate competitive and market data. Pricing changes, catalog expansion, development of new products, and where to go to market are just a few. However, these decisions are only as good as the insights derived from the data. If the data is made up of inaccurate or low-quality inputs, the outputs will also be low-quality.

    Managing eCommerce data at scale gets more complex every year. There are more market entrants, retailers, and copy-cats trying to sell similar or knock-off products. There are millions of SKUs from thousands of retailers in multiple markets. Not only that, the data is constantly changing. Amazon may add a new subcategory definition in an existing space, or Staples might decide to branch out into a new industry like “snack foods for the office”, an established brand might introduce new sizing options in their apparel, or shrinkflation might decrease the size of a product.

    Given this, it is imperative that conventional data collection and validation methods need to be revised. Teams that rely on spreadsheets and manual auditing processes can’t keep up with the scale and speed of change. An algorithm that once could match products easily needs to be updated when trends, categories, or terminology change.

    With SKU proliferation, visually matching product images against the competition becomes impossible. Knowing where to look for comprehensive data becomes impossible with so many new sellers in the market. Luckily, technology has advanced to a place where manual intervention isn’t the main course of action.

    Advanced AI capabilities, like DataWeave’s, tackle these challenges to help gather, categorize, and extract insights that drive impactful business decisions. It performs the millions of actions that your team can’t accomplish with greater accuracy and in near real-time.

    Improving the Accuracy of Product Matching

    Image Matching for Data Quality

    DataWeave’s product matching capabilities rely on an ensemble of text and image-based models with built-in loss functions to determine confidence levels in all insights. These loss functions measure precision and recall. They help in determining how accurate – both in terms of correctness and completeness – the results are so the system can learn and improve over time. The solution’s built-in scoring function provides a confidence metric that brands and retailers can rely on.

    The product matching engine is configurable based on the type of products that we are matching. It uses a “pipelined mode” that first focuses on recall or coverage by maximizing the search space for viable candidates, followed by mechanisms to improve the precision.

    How ‘Embeddings’ Enhance Scoring

    Embeddings are like digital fingerprints. They are dense vector representations that capture the essence of a product in a way that makes it easy to identify similar products. With embeddings, we can codify a more nuanced understanding of the varied relationships between different products. Techniques used to create good embeddings are generic and flexible and work well across product categories. This makes it easier to find similarities across products even with complex terminology, attributes, and semantics.

    These along with advanced scoring mechanisms used across DataWeave’s eCommerce offerings provide the foundation for:

    • Semantic Analysis: Embeddings identify subtle patterns and meanings in text and image data to better align with business contexts.
    • Multimodal Integration: A comprehensive representation of each SKU is created by incorporating embeddings from both text (product descriptions) and images or videos (product visuals)
    • Anomaly Detection: AI models leverage embeddings to identify outliers and inconsistencies to improve the overall score accuracy.
    DataWeave's AI Tech Stack

    Vector Databases for Enhanced Accuracy

    Vector databases play a central role in DataWeave’s AI ecosystem. These databases help with better storage, retrieval, and scoring of embeddings and serve to power real-time applications such as Verification. This process helps pinpoint the closest matches for products, attributes, or categories with the help of similarity algorithms. It can even operate when there is incomplete or noisy data. After identification, the system prioritizes data that exhibits high semantic alignment so that all recommendations are high-quality and relevant.

    Evolution of Embeddings and Scoring: A Multimodal Perspective

    Product listings undergo daily visual and text changes. DataWeave takes a multimodal approach in its AI to ensure that any content shown on a listing is accounted for, including visuals, videos, contextual signals, and text. DataWeave is continually evolving its embedding and scoring models to align with industry advancements and always works within an up-to-date context.

    DataWeave’s AI framework can:

    • Handle Diverse Data Types: The framework captures a holistic view of the digital shelf by integrating insights from multiple sources.
    • Improve Matching Precision: Sophisticated scoring methods refine the accuracy of matches so that brands and retailers can trust the competitive intelligence.
    • Scale Across Markets: Additional, expansive datasets are easy for DataWeave’s capabilities, meaning brands and retailers can scale across markets without pausing.

    Quantified Improvements: Model Accuracy and Stats

    • Since we deployed LLMs and CLIP Embeddings, Product Matching accuracy improved by > 15% from the previous baseline numbers in categories such as Home Improvement, Fashion, and CPG.
    • High precision in certain categories such as Electronics and Fashion. Upwards of 85%.
    • Close to 90% of matches are auto-processed (auto-verified or auto-rejected).
    • Attribute tagging accuracy > 75% and significant improvement for the top 5 categories.

    Business Use Case: Multimodal Matching for Price Leadership

    For example, if you’re a retailer selling consumer electronics, you probably want to maintain your price leadership across your key markets during peak times like Black Friday Cyber Monday. Doing so is a challenge, as all your competitors are changing prices several times a day to steal your sales. To get ahead of them, this retailer could use DataWeave’s multimodal embedding-based scoring framework to:

    • Detect Discrepancies: Isolate SKUs with price mismatches with your competition and take action before revenue is lost.
    • Optimize Coverage: Establish a process to capture complete data across the competition so you can avoid knowledge gaps.
    • Enable Timely Decisions: Address the ‘low-hanging fruit’ by prioritizing products that need pricing adjustments based on confidence scores on high-impact products. Leverage confidence metrics to prioritize pricing adjustments for high-impact products.

    This approach helps retailers stay competitive even as eCommerce evolves around us. By acting fast on complete and reliable data, they can earn and sustain their competitive advantage.

    DataWeave’s AI-Driven Data Quality Framework

    Let’s look at how our AI can gather the most comprehensive data and output the highest-quality insights. Our framework evaluates three critical dimensions:

    • Accuracy: “Is my data correct?” – Ensuring reliable product matches and attribute tracking
    • Coverage: “Do I have the complete picture?” – Maintaining comprehensive market visibility
    • Freshness: “Is my data recent?” – Guaranteeing timely and current market insights
    The 3 pillars to gauge data quality at DataWeave

    Scoring Data Quality

    To maintain the highest levels of data quality, we rely on a robust scoring mechanism across our solutions. Every dataset that is evaluated is done so based on several key parameters. These can include things like accuracy, consistency, timeliness, and completeness of data. Scores are dynamically updated as new data flows in so that insights can be acted upon.

    • Accuracy: Compare gathered data with multiple trusted sources to reduce discrepancies.
    • Consistency: Detect and rectify variations or contradictions across the data with regular audits.
    • Timeliness: Scoring emphasizes data recency, especially for fast-changing markets like eCommerce.
    • Completeness: Ensure all essential data points are included and gaps in coverage are highlighted by analysis.

    Apart from this, we also leverage an evolved quality check framework:

    DataWeave's Data Quality Check framework

    Statistical Process Control

    DataWeave implements a sophisticated system of statistical process control that includes:

    • Anomaly Detection: Using advanced statistical techniques to identify and flag outlier data, particularly for price and stock variations
    • Intelligent Alerting: Automated system for notifying stakeholders of significant deviations
    • Continuous Monitoring: Real-time tracking of data patterns and trends
    • Error Correction: Systematic approach to addressing and rectifying data discrepancies

    Transparent Quality Assurance

    The platform provides complete visibility into data quality through:

    • Comprehensive Data Transparency & Statistics Dashboard: Offering detailed insights into match performance and data freshness
    • Match Distribution Analysis: Tracking both exact and similar matches across retailers and locations as required
    • Product Tracking Metrics: Visibility into the number of products being monitored
    • Autonomous Audit Mechanisms: Giving customers access to cached product pages for transparent, on-demand verification

    Human-in-the-Loop Validation (Véracité)

    DataWeave’s Véracité system combines AI capabilities with human expertise to ensure unmatched accuracy:

    • Expert Validation: Product category specialists who understand industry-specific similarity criteria
    • Continuous Learning: AI models that evolve through ongoing expert feedback
    • Adaptive Matching: Recognition that similarity criteria can vary by category and change over time
    • Detailed Documentation: Comprehensive reasoning for product match decisions

    Together, these elements create a robust framework that delivers accurate, complete, and relevant product data for competitive intelligence. The system’s combination of automated monitoring, statistical validation, and human expertise ensures businesses can make decisions based on reliable, high-quality data.

    In Conclusion

    DataWeave’s AI-driven approach to data quality and coverage empowers retailers and brands to navigate the complexities of eCommerce with confidence. By leveraging advanced techniques such as multimodal embeddings, vector databases, and advanced scoring functions, businesses can ensure accurate, comprehensive, and timely competitive intelligence. These capabilities enable them to optimize pricing, improve product visibility, and stay ahead in an ever-evolving market. As AI continues to refine product matching and data validation processes, brands can rely on DataWeave’s technology to eliminate inefficiencies and drive smarter, more profitable decisions.

    The evolution of AI in competitive intelligence is not just about automation—it’s about precision, scalability, and adaptability. DataWeave’s commitment to high data quality standards, supported by statistical process controls, transparent validation mechanisms, and human-in-the-loop expertise, ensures that insights remain actionable and trustworthy. In a digital landscape where data accuracy directly impacts profitability, investing in AI-powered solutions like DataWeave’s is not just an advantage—it’s a necessity for sustained eCommerce success.

    To learn more, reach out to us today or email us at contact@dataweave.com.

  • Enterprise Data Security at DataWeave: Empowering Smarter Decisions with Seamless, Secure Data Management and Integration

    Enterprise Data Security at DataWeave: Empowering Smarter Decisions with Seamless, Secure Data Management and Integration

    At DataWeave, data security isn’t just about compliance—it’s about enabling peace of mind and better decision-making for our customers. Our customers rely on us not just for competitive and market intelligence but also for the seamless integration of critical data sources into their decision-making frameworks. To achieve this, we have built a security-first infrastructure that ensures organizations can confidently leverage both external and internal data without compromising privacy or protection.

    Secure Data Integration: The Foundation of Smarter Decisions

    Effective decision-making in today’s digital commerce landscape depends on combining multiple data sources—including first-party customer data, pricing intelligence, and business rules—into a unified framework. However, without the right security measures in place, businesses often struggle to operationalize this data effectively.

    At DataWeave, we eliminate this challenge by offering:

    • Integration with Leading Data Storage Solutions: Our platform seamlessly connects with data lakes and warehouses like AWS S3 and Snowflake, ensuring that businesses can easily ingest and analyze our data in real time.
    DataWeave's Data Security Framework
    • Support for Sandboxed Environments & Data Clean Rooms: Organizations can securely merge internal and external datasets without compromising confidentiality, unlocking deeper insights for pricing and business strategies.
    • Automated Data Ingestion & Management: We simplify the process of integrating first-party data alongside competitive insights, allowing customers to focus on execution rather than infrastructure management.

    Our Purpose-Built Security Framework

    Handling millions of data points daily demands a security framework that is not only robust but also scalable and adaptable to evolving threats. DataWeave’s multi-tenant architecture ensures seamless data security without compromising operational efficiency.

    • Multi-Tenant Architecture: Our system allows multiple customers to share the same application infrastructure while maintaining complete data isolation and security.
      • Tenants share infrastructure and computing resources but remain logically isolated.
      • Application-level controls ensure privacy while maximizing cost efficiency.
      • Centralized updates, maintenance, and easy scalability for new tenants.
    • End-to-End Encryption & Access Controls: Every piece of data is encrypted both in transit and at rest. Role-based access controls (RBAC) restrict visibility to only authorized personnel, ensuring minimal risk of unauthorized data access.

    Active Monitoring & Automated Compliance Management: We leverage automated access controls that adjust permissions dynamically as organizational roles evolve, ensuring that compliance is continuously maintained.

    Certifications That Inspire Confidence

    Data security is at the core of everything we do. Our compliance with the highest industry standards ensures that businesses can trust us with their sensitive data.

    SOC 2 Type II Certification: DataWeave’s SOC 2 compliance is a testament to our commitment to stringent security protocols. This certification guarantees that we adhere to strict standards in data protection, availability, and confidentiality.

    We implement a phased approach to security improvement:

    • Prioritizing Critical Systems: To maximize impact, we prioritized systems that had the highest data security relevance and expanded the coverage thereafter. By addressing these priority areas, we were able to make meaningful security improvements early in the process.
    • Automating Monitoring and Compliance: Partnering with Sprinto streamlined the compliance journey by automating key processes. This included real-time monitoring of our cloud environments, automated generation of audit-ready evidence, and integration with critical systems like AWS, Bitbucket, and Jira. These enhancements ensured efficient management of compliance requirements while reducing the burden on our teams.
    SOC 2 Compliance at DataWeave
    • Fostering a Culture of Shared Responsibility: We conducted organization-wide training sessions to embed compliance as a shared responsibility across all teams. By educating employees on the importance of security practices and providing them with the tools to manage compliance autonomously, we established a security-first mindset throughout the company.

    This systematic method allowed us to deliver immediate improvements while aligning long-term practices with industry’s best standards.

    What This Means for Our Customers

    By combining robust security with seamless data integration, DataWeave empowers businesses to:

    • Optimize Price Management & Modelling: With secure access to real-time data, organizations can make informed pricing decisions that enhance profitability and market competitiveness.
    • Run Advanced Simulations & Testing: Reliable, secure data enables businesses to model various pricing and assortment strategies before implementation, reducing risks and maximizing returns.
    • Uncompromised Data Security: SOC 2 Type II compliance ensures stringent protocols to protect your data at every stage.
    • Simplified Vendor Processes: Verified security certifications reduce friction during due diligence and onboarding, making it easier to partner with us.
    • Aligned Standards: Our adherence to industry benchmarks reflects our commitment to meeting your expectations as a trusted technology partner.
    • Scalable Operations: Expand across regions while maintaining full confidence in data privacy and security.
    • Secure Collaboration: Share insights across teams with tools designed to protect sensitive information.

    Our customers are increasingly looking to integrate their internal datasets with the external competitive intelligence provided by DataWeave. This can be a complex and risky process without the right security measures in place. We remove these roadblocks by providing a secure, scalable infrastructure designed to help businesses unify data without security concerns.

    By ensuring seamless compatibility with key data storage platforms, such as Snowflake and AWS S3, we enable organizations to consolidate valuable first-party data with timely market insights. This integration empowers businesses to refine their pricing, assortment, and digital shelf strategies, thereby driving superior customer experiences—without the headaches of data security risks.

    Security remains a top priority in everything we do. Our SOC 2 Type II-certified framework enforces rigorous encryption, access controls, and real-time compliance monitoring. We take on the burden of data security so our customers can focus on innovation and growth.

    With DataWeave, businesses can confidently leverage secure data-driven decision-making to unlock new opportunities, optimize operations, and scale without compromise.

    To learn more, write to us at contact@dataweave.com or request a consultation here.

  • Black Friday 2024 in Canada: Insights on Consumer Electronics and Home & Furniture

    Black Friday 2024 in Canada: Insights on Consumer Electronics and Home & Furniture

    Black Friday and Cyber Monday are major retail events in Canada, with 43% and 29% of the population making purchases during these sales respectively, according to a YouGov report. Consumer electronics continue to lead the Canadian retail market during these events, with 55% of surveyed shoppers choosing to buy tech products on Black Friday. Household appliances come in second, with 25% of shoppers opting for these items, while 18% prefer to shop for furniture deals.

    These statistics highlight the importance of delivering value during the Thanksgiving sales week. Retailers must cater to shoppers’ expectations with competitive pricing, attractive deals, and a seamless shopping experience. So, what unique offerings did Canadian retailers present to shoppers this season?

    To understand the pricing and discount dynamics during BFCM 2024 in Canada, DataWeave analyzed discounts across leading consumer electronics and home & furniture retailers. Using our AI-powered pricing intelligence platform, we analyzed 37,108 SKUs across these categories for major retailers including Amazon, Walmart, Best Buy, Home Depot, and Canadian Tire from the 10th to 29th November. We focused on the top 500 products ranked for each search keyword on each retail site, using targeted terms aligned with categories like “sofa” and “wearables”.

    In the following insights, the Absolute Discount represents the reduction of the selling price compared to the Manufacturer’s Suggested Retail Price (MSRP). The Additional Discount reflects how much lower the selling price is during Black Friday compared to its price a week before the sale. This metric reveals the actual or effective value of the sale event, beyond the standard discounts typically offered.

    Also check out our detailed analysis of discounts and pricing for the consumer electronics, apparel, health & beauty, grocery, and home & furniture categories across major US retailers this Black Friday.

    Consumer Electronics

    Retailers in Focus

    Consumer electronics saw robust participation from major retailers, with Amazon, Best Buy, and Walmart leading the charge. Here’s how they stacked up in terms of discounts:

    Pricing Trends Across Leading Consumer Electronics Retailers in Canada - Black Friday Cyber Monday 2024
    • Best Buy emerged as the frontrunner in absolute discounts at 31.2%, while Amazon impressed with a notable 19.7% additional discount, indicating a strong Black Friday-specific markdown strategy.
    • Walmart offered steady competition, particularly in audio and video products, which reached an average absolute discount of 37.2%. However, it’s average additional discount was only 3.1%, indicating muted BFCM-specific price reductions in this category.

    Subcategory Insights

    Diving deeper into consumer electronics subcategories, we observed varied discounting strategies.

    Pricing Trends Across Leading Canadian Consumer Electronics Retailer Subcategories - Black Friday Cyber Monday 2024
    • Audio & Video stood out as the most discounted subcategory, with Walmart leading at 37.2%.
    • In Wearables, Walmart again took the top spot with 36.4%, while Amazon offered higher additional discounts (22.4%).
    • Discounting for computers and gaming was less aggressive, highlighting strategic pricing to maintain profitability in these high-demand segments.

    Brand Performance

    Brand-level data highlighted how key players used Black Friday to drive visibility and sales.

    Pricing Trends Across Leading Canadian Consumer Electronics Brands - Black Friday Cyber Monday 2024
    • Dell led in average absolute discounts (36.7%) followed by Samsung at 36.68%
    • Audio brand JBL offered significant absolute discounts at 35.9%.
    • Apple and Lenovo offered comparatively fewer discounts but maintained strong visibility, as seen in their increase in the Share of Search during the sale period.
    Visibility Trends Across Leading Canadian Consumer Electronics Brands - Share of Search - Black Friday Cyber Monday 2024
    • MSI (laptop brand) and Bose (audio and earphone brand) experienced significant increases in visibility, with Share of Search increases of 5% and 3.6%, respectively.
    • Notably, HP faced a decline (-3.2%) in the Share of Search, suggesting missed opportunities to align promotions with consumer interest.

    Home & Furniture

    Retailers in Focus

    The home and furniture category saw competitive discounting, with Walmart, Canadian Tire, and Home Depot vying for consumer attention.

    Black Friday - Cyber Monday Trends Across Leading Canadian Home & Furniture Retailers
    • Walmart took the lead with the highest absolute discounts at 36.8%. The retailer’s additional discounts were more conservative at 3.6%. This is similar to their discount levels in Consumer Electronics.
    • Canadian Tire offered stiff competition, providing 31.6% absolute discounts and 25% additional discounts.
    • Home Depot matched its absolute and additional discounts, maintaining consistency at 24.1%.

    Subcategory Insights

    Home and furniture subcategories revealed targeted discount strategies.

    Black Friday - Cyber Monday Trends Across Leading Home & Furniture Subcategories - Canada
    • Bedding emerged as the most discounted subcategory at Walmart (50.6%) and Canadian Tire (35.3%).
    • Kitchenware saw competitive pricing, with Walmart leading at 42.9%, followed by Canadian Tire at 33.9%.
    • Canadian Tire focused on lighting, offering the highest absolute discounts in this subcategory (38.2%)

    Brand Performance

    Brand-level analysis revealed stark contrasts in discounting approaches.

    Black Friday - Cyber Monday Trends Across Leading Home & Furniture Brands - Canada
    • Furniture brands Homcom led in absolute discounts (36.4%), while South Shore stood out with the highest additional discounts (30.2%).
    • Value-oriented brands like furnishings brand Mainstays and mattress and bedding brand Zinus offered more modest discounts, focusing on consistent affordability.
    Black Friday - Cyber Monday Trends Across Leading Canadian Home & Furniture Brands - Share of Search and Visibility
    • Zinus (mattresses and sofa brand) experienced a significant 7.9% increase in the Share of Search, driven by aggressive promotions.
    • Home furnishings brands like Costway and Safavieh faced declines, reflecting the importance of aligning promotional strategies with consumer expectations.

    Insights for Retailers and Brands

    This Black Friday, Canadian retailers effectively balanced deep discounts with category-specific strategies to maximize sales. However, the fluctuating Share of Search highlights the critical need for brands to align promotions with consumer interest.

    For brands and retailers looking to stay ahead of the competition, DataWeave’s pricing intelligence platform offers unparalleled insights to refine discounting strategies and boost visibility. Contact us to learn how we can help you stay competitive in this dynamic retail landscape.

  • Early Black Friday Deals Analyzed: How Top Retailers Stack Up on Discounts

    Early Black Friday Deals Analyzed: How Top Retailers Stack Up on Discounts

    Black Friday, once confined to a single weekend, has evolved into a shopping season that now stretches well before Thanksgiving. With inflation hovering around 3% and consumer confidence showing signs of recovery, retailers are adapting their promotional calendars to capture early-bird shoppers and maintain a competitive edge.

    Major retailers, including Amazon, Walmart, Target, and Best Buy, have capitalized on this trend by launching promotions weeks in advance, signaling the traditional holiday rush is now a month-long event. At DataWeave, we put these deals under a microscope.

    Our Methodology

    Using DataWeave’s advanced, AI-powered pricing intelligence platform, we tracked early Black Friday deals across Consumer Electronics, Home & Furniture, Health & Beauty, and Apparel categories. We monitored dedicated Black Friday deal pages on Amazon, Walmart, Target, Best Buy, Nordstrom, Neiman Marcus, and Sephora to gather and analyze discount data a week prior to Black Friday weekend.

    Who’s Offering the Best Deals Across Categories?

    Our pre- Black Friday analysis reveals a clear pattern of premium brands offering deeper discounts across categories ahead of the holiday. Here are some key findings around retail players:

    • Walmart emerges as the most aggressive discounter across categories, leading in Health & Beauty (57.07%), Apparel (48.97%), and Consumer Electronics (43.35%).
    • Amazon maintains consistent but lower discounts (28-29%) across categories, suggesting potential deeper cuts ahead.
    • Best Buy and Sephora, both category specialists, play it conservative compared to mass retail players.

    Let’s look at each category more closely to get a detailed snapshot of the deals this Thanksgiving week:

    Health & Beauty

    Our analysis reveals that it’s not electronics, but the health & beauty category that leads with the widest discount range pre Black Friday, making it the category to watch out for.

    • Walmart takes the lead with an aggressive 57.1% average discount in this category, capitalizing on its value-oriented reputation.
    • Beauty specialist Sephora holds modest beauty discounts (32.81%) compared to other retailers.
    • Amazon offers the broadest range of SKUs (571) in the category.
    Avg. Discounts Across Retailers Pre Black Friday 2024 - Health & Beauty

    Among the health & beauty brands we analyzed, cosmetics brand Tarte and viral K-Beauty skincare brand COSRX stand out with discounts above 40%, appealing to cost-conscious beauty enthusiasts.

    Brands with Highest Avg. Discounts Before Black Friday 2024 - Health & Beauty

    Consumer Electronics

    Our pre- Black Friday analysis reveals interesting insights about consumer electronics deals this season.

    • Walmart, once again, emerges as the frontrunner in the category with 43.4% average discounts.
    • Best Buy plays it conservative in electronics (30.75%), despite being a category specialist, but offers the most extensive SKU coverage (3030).
    • Amazon’s consistent 29.7% discount across 1,749 SKUs suggests they’re probably holding back their best deals for Prime members during Black Friday.
    Avg. Discounts Across Retailers Pre Black Friday 2024 - Consumer Electronics

    Brand-specific data for the category reveals significant deals on Speck (48.07%) and smart TV brand Insignia (39.22%), making accessories and mid-tier electronics attractive for early shoppers. Core computing (HP at 32.14%) and electronics brands maintain more conservative discounts. It remains to be seen if this changes on Black Friday or Cyber Monday.

    Brands with Highest Avg. Discounts Before Black Friday 2024 - Consumer Electronics

    Apparel

    Our analysis of the apparel category reveals several highlights:

    • In the apparel category too, Walmart dominates with an impressive 49% average discount, effectively targeting price-sensitive shoppers in the fashion segment.
    • Nordstrom and Neiman Marcus, both known for apparel, offer significant discounts at 43.2% and 37.8% respectively.
    • Amazon’s expansive SKU coverage (1344) is countered by a modest 29.5% discount, showing its focus on variety over depth of discounts.
    Avg. Discounts Across Retailers Pre Black Friday 2024 - Apparel

    Premium fashion brands dominate the highest discounts this Black Friday in the apparel category. Vince Camuto leads with over 45.1% average discount. Notably, Levi and Nike’s aggressive 44.43% and 43.50% discounts suggests significant inventory positions or intent to capture market share.

    Brands with Highest Avg. Discounts Before Black Friday 2024 - Apparel

    Home & Furniture

    Our analysis reveals an interesting trend across the category.

    • In the home & furniture category too, Walmart leads at 41.8% average discounts. Target follows closely, but with significantly lesser SKUs on offer.
    • Amazon’s 28.1% discount, though the lowest among major players, spans a substantial 1,982 SKUs, reinforcing its position as a marketplace for diverse needs.
    Avg. Discounts Across Retailers Pre Black Friday 2024 - Home & Furniture

    Top 3 Products With the Highest Discounts Across Retailers

    To provide a clearer picture of the early Black Friday landscape, we analyzed the top 3 products with the most substantial discounts in consumer electronics and health & beauty categories. These insights highlight how retailers are leveraging strategic discounts on high-value items to attract early shoppers.

    Top Discounted Products in Consumer Electronics

    Premium TVs dominate the discount scene, with LG’s 83″ OLED offering up to 44.5% off on Amazon, closely followed by a 44.4% discount on Best Buy, showcasing aggressive competition. The same product has much lower discounting on Walmart, but notably, the product is retailed at $3999.9, at least $1000 less than other retailers, highlighting Walmart’s commitment to offering lowest prices.

    Products With Highest Discounts Pre Black Friday 2024 - Consumer Electronics - TVs
    Products With Highest Discounts Pre Black Friday 2024 - Consumer Electronics - Playstation
    Products With Highest Discounts Pre Black Friday 2024 - Consumer Electronics - Digital Cameras

    Gaming consoles, like the PlayStation 5 Slim Bundle, show moderate discounts (ranging from 15% on Walmart and Target to 25% at Best Buy), appealing to tech-savvy shoppers.

    Notable competition is evident in price matching across major retailers, particularly in TVs and high-value electronics like the Nikon Z 8 camera, where Walmart offers the deepest discount at 13.75%, edging past Amazon and Best Buy.

    Top Discounted Products in Health & Beauty

    Viral skincare staples like Tatcha’s Water Cream show tight discounting consistency, with Walmart offering 19.47% off compared to Amazon’s 20% and Sephora’s 20.83%.

    Products With Highest Discounts Pre Black Friday 2024 - Health & Beauty - Tatcha Water Cream
    Products With Highest Discounts Pre Black Friday 2024 - Health & Beauty - Olaplex Hair Oil
    Products With Highest Discounts Pre Black Friday 2024 - Health & Beauty - Yves Saint Laurent Satin Lipstick

    Trending haircare brand Olaplex displays greater disparity, with Walmart leading with a 33.33% discount, surpassing Amazon and Sephora. Luxury brand, Yves Saint Laurent’s Satin Lipstick is one of the highest discounted items across retailers.

    Looking Ahead

    Our analysis suggests that while some early deals offer genuine value, particularly in premium beauty and high-end electronics, many retailers might be holding their best discounts for Black Friday.

    For shoppers, the key is being selective: jump on premium brand discounts now (since they’re likely to remain the same though the weekend), but wait on mid-range electronics and home goods where better deals are likely to emerge on Black Friday or Cyber Monday.

    For retailers, the imperative is clear: dynamic pricing intelligence is crucial for maintaining a competitive edge while protecting margins. Competitive insights will be critical as the holiday season progresses to balance market share against profitability.

    Stay tuned for our Black Friday Cyber Monday analysis next week, where we’ll track how these early discounts compare to the main event’s deals!

  • Redefining Product Attribute Tagging With AI-Powered Retail Domain Language Models

    Redefining Product Attribute Tagging With AI-Powered Retail Domain Language Models

    In online retail, success hinges on more than just offering quality products at competitive prices. As eCommerce catalogs expand and consumer expectations soar, businesses face an increasingly complex challenge: How do you effectively organize, categorize, and present your vast product assortments in a way that enhances discoverability and drives sales?

    Having complete and correct product catalog data is key. Effective product attribute tagging—a crucial yet frequently undervalued capability—helps in achieving this accuracy and completeness in product catalog data. While traditional methods of tagging product attributes have long struggled with issues of scalability, consistency, accuracy, and speed, a new thinking and fundamentally new ways of addressing these challenges are getting established. These follow from the revolution brought in Large Language Models but they fashion themselves as Small Language Models (SLM) or more precisely as Domain Specific Language Models. These can be potentially considered foundational models as they solve a wide variety of downstream tasks albeit within specific domains. They are a lot more efficient and do a much better job in those tasks compared to an LLM. .

    Retail Domain Language Models (RLMs) have the potential to transform the eCommerce customer journey. As always, it’s never a binary choice. In fact, LLMs can be a great starting point since they provide an enhanced semantic understanding of the world at large: they can be used to mine structured information (e.g., product attributes and values) out of unstructured data (e.g., product descriptions), create baseline domain knowledge (e.g, manufacturer-brand mappings), augment information (e.g., image to prompt), and create first cut training datasets.

    Powered by cutting-edge Generative AI and RLMs, next-generation attribute tagging solutions are transforming how online retailers manage their product catalog data, optimize their assortment, and deliver superior shopping experiences. As a new paradigm in search emerges – based more on intent and outcome, powered by natural language queries and GenAI based Search Agents – the capability to create complete catalog information and rich semantics becomes increasingly critical.

    In this post, we’ll explore the crucial role of attribute tagging in eCommerce, delve into the limitations of conventional tagging methods, and unveil how DataWeave’s innovative AI-driven approach is helping businesses stay ahead in the competitive digital marketplace.

    Why Product Attribute Tagging is Important in eCommerce

    As the eCommerce landscape continues to evolve, the importance of attribute tagging will only grow, making it a pertinent focus for forward-thinking online retailers. By investing in robust attribute tagging systems, businesses can gain a competitive edge through improved product comparisons, more accurate matching, understanding intent, and enhanced customer search experiences.

    Taxonomy Comparison and Assortment Gap Analysis

    Products are categorized and organized differently on different retail websites. Comparing taxonomies helps in understanding focus categories and potential gaps in assortment breadth in relation to one’s competitors: missing product categories, sizes, variants or brands. It also gives insights into the navigation patterns and information architecture of one’s competitors. This can help in making search and navigation experience more efficient by fine tuning product descriptions to include more attributes and/or adding additional relevant filters to category listing pages.

    For instance, check out the different Backpack categories on Amazon and Staples in the images below.

    Product Names and Category Names Differ on Different eCommerce Platforms - Here's an Amazon Example
    Product Names and Category Names Differ on Different eCommerce Platforms - Here's a Staples Example

    Or look at the nomenclature of categories for “Pens” on Amazon (left side of the image) and Staples (right side of the image) in the image below.

    Product Names and Category Names Differ on Different eCommerce Platforms -Here's how Staples Vs. Amazon Categories look for Pens

    Assortment Depth Analysis

    Another big challenge in eCommerce is the lack of standardization in retailer taxonomy. This inconsistency makes it difficult to compare the depth of product assortments across different platforms effectively. For instance, to categorize smartphones,

    • Retailer A might organize it under “Electronics > Mobile Phones > Smartphones”
    • Retailer B could use “Technology > Phones & Accessories > Cell Phones”
    • Retailer C might opt for “Consumer Electronics > Smartphones & Tablets”

    Inconsistent nomenclature and grouping create a significant hurdle for businesses trying to gain a competitive edge through assortment analysis. The challenge is exacerbated if you want to do an in-depth assortment depth analysis for one or more product attributes. For instance, look at the image below to get an idea of the several attribute variations for “Desks” on Amazon and Staples.

    With Multiple Attributes Named in a Variety of Ways, Attribute Tagging is Essential to Ensure Accurate Product Matching

    Custom categorization through attribute tagging is essential for conducting granular assortment comparisons, allowing companies to accurately assess their product offerings against those of competitors.

    Enhancing Product Matching Capabilities

    Accurate product matching across different websites is fundamental for competitive pricing intelligence, especially when matching similar and substitute products. Attribute tagging and extraction play a crucial role in this process by narrowing down potential matches more effectively, enabling matching for both exact and similar products, and tagging attributes such as brand, model, color, size, and technical specifications.

    For instance, when choosing to match similar products in the Sofa category for 2-3 seater sofas from Wayfair and Overstock, tagging attributes like brand, color, size, and more is a must for accurate comparisons.

    Attribute Tagging for Home & Furniture Categories Like Sofas Helps Improve Matching Accuracy
    Attribute Tagging for Home & Furniture Categories Like Sofas Helps Improve Matching Accuracy

    Taking a granular approach not only improves pricing strategies but also helps identify gaps in product offerings and opportunities for expansion.

    Fix Content Gaps and improve Product Detail Page (PDP) Content

    Attribute tagging plays a vital role in enhancing PDP content by ensuring adherence to brand integrity standards and content compliance guidelines across retail platforms. Tagging attributes allows for benchmarking against competitor content, identifying catalog gaps, and enriching listings with precise details.

    This strategic tagging process can highlight missing or incomplete information, enabling targeted optimizations or even complete rewrites of PDP content to improve discoverability and drive conversions. With accurate attribute tagging, businesses can ensure each product page is fully optimized to capture consumer attention and meet retail standards.

    Elevating the Search Experience

    In today’s online retail marketplace, a superior search experience can be the difference between a sale and a lost customer. Through in-depth attribute tagging, vendors can enable more accurate filtering to improve search result relevance and facilitate easier product discovery for consumers.

    By integrating rich product attributes extracted by AI into an in-house search platform, retailers can empower customers with refined and user-friendly search functionality. Enhanced search capabilities not only boost customer satisfaction but also increase the likelihood of conversions by helping shoppers find exactly what they’re looking for more quickly and with minimal effort.

    Pitfalls of Conventional Product Tagging Methods

    Traditional methods of attribute tagging, such as manual and rule-based systems, have been significantly enhanced by the advent of machine learning. While these approaches may have sufficed in the past, they are increasingly proving inadequate in the face of today’s dynamic and expansive online marketplaces.

    Scalability

    As eCommerce catalogs expand to include thousands or even millions of products, the limitations of machine learning and rule-based tagging become glaringly apparent. As new product categories emerge, these systems struggle to keep pace, often requiring extensive revisions to existing tagging structures.

    Inconsistencies and Errors

    Not only is reliance on an entirely human-driven tagging process expensive, but it also introduces a significant margin for error. While machine learning can automate the tagging process, it’s not without its limitations. Errors can occur, particularly when dealing with large and diverse product catalogs.

    As inventories grow more complex to handle diverse product ranges, the likelihood of conflicting or erroneous rules increases. These inconsistencies can result in poor search functionality, inaccurate product matching, and ultimately, a frustrating experience for customers, drawing away the benefits of tagging in the first place.

    Speed

    When product information changes or new attributes need to be added, manually updating tags across a large catalog is a time-consuming process. Slow tagging processes make it difficult for businesses to quickly adapt to emerging market trends causing significant delays in listing new products, potentially missing crucial market opportunities.

    How DataWeave’s Advanced AI Capabilities Revolutionize Product Tagging

    Advanced solutions leveraging RLMs and Generative AI offer promising alternatives capable of overcoming these challenges and unlocking new levels of efficiency and accuracy in product tagging.

    DataWeave automates product tagging to address many of the pitfalls of other conventional methods. We offer a powerful suite of capabilities that empower businesses to take their product tagging to new heights of accuracy and scalability with our unparalleled expertise.

    Our sophisticated AI system brings an advanced level of intelligence to the tagging process.

    RLMs for Enhanced Semantic Understanding

    Semantic Understanding of Product Descriptions

    RLMs analyze the meaning and context of product descriptions rather than relying on keyword matching.
    Example: “Smartphone with a 6.5-inch display” and “Phone with a 6.5-inch screen” are semantically similar, though phrased differently.

    Attribute Extraction

    RLMs can identify important product attributes (e.g., brand, size, color, model) even from noisy or unstructured data.
    Example: Extracting “Apple” as a brand, “128GB” as storage, and “Pink” as the color from a mixed description.

    Identifying Implicit Relationships

    RLMs find implicit relationships between products that traditional rule-based systems miss.
    Example: Recognizing that “iPhone 12 Pro” and “Apple iPhone 12” are part of the same product family.

    Synonym Recognition in Product Descriptions

    Synonym Matching with Context

    RLMs identify when different words or phrases describe the same product.
    Examples: “Sneakers” = “Running Shoes”, “Memory” = “RAM” (in electronics)
    Even subtle differences in wording, like “rose gold” vs “pink” are interpreted correctly.

    Overcoming Brand-Specific Terminology

    Some brands use their own terminologies (e.g., “Retina Display” for Apple).
    RLMs can map proprietary terms to more generic ones (e.g., Retina Display = High-Resolution Display).

    Dealing with Ambiguities

    RLMs analyze surrounding text to resolve ambiguities in product descriptions.
    Example: Resolving “charger” to mean a “phone charger” when matched with mobile phones.

    Contextual Understanding for Improved Accuracy and Precision

    By leveraging advanced natural language processing (NLP), DataWeave’s AI can process and understand the context of lengthy product descriptions and customer reviews, minimizing errors that often arise at human touch points. The solution processes and interprets information to extract key information to dramatically improve the overall accuracy of product tags.

    It excels at grasping the subtle differences between similar products, sizes, colors and identifying and tagging minute differences between items, ensuring that each product is uniquely and accurately represented in a retailer’s catalog.

    This has a major impact on product and similarity-based matching that can even help optimize similar and substitute product matching to enhance consumer search. At the same time, our AI can understand that the same term might have different meanings in various product categories, adapting its tagging approach based on the specific context of each item.

    This deep comprehension ensures that even nuanced product attributes are accurately captured and tagged for easy discoverability by consumers.

    Case Study: Niche Jewelry Attributes

    DataWeave’s advanced AI can assist in labeling the subtle attributes of jewelry by analyzing product images and generating prompts to describe the image. In this example, our AI identifies the unique shapes and materials of each item in the prompts.

    The RLM can then extract key attributes from the prompt to generate tags. This assists in accurate product matching for searches as well as enhanced product recommendations based on similarities.

    DataWeave's AI assists in extracting contextual attributes for accuracy in product matching

    This multi-model approach provides the flexibility to adapt as product catalogs expand while remaining consistent with tagging to yield more robust results for consumers.

    Unparalleled Scalability

    DataWeave can rapidly scale tagging for new categories. The solution is built to handle the demands of even the largest eCommerce catalogs enabling:

    • Effortless management of extensive product catalogs: We can process and tag millions of products without compromising on speed or accuracy, allowing businesses to scale without limitations.
    • Automated bulk tagging: New product lines or entire categories can be tagged automatically, significantly reducing the time and resources required for catalog expansion.

    Normalizing Size and Color in Fashion

    Style, color, and size are the core attributes in the fashion and apparel categories. Style attributes, which include design, appearance, and overall aesthetics, can be highly specific to individual product categories.

    Normalizing Size and Color in Fashion for Product Matching

    Our product matching engine can easily handle color and sizing complexity via our AI-driven approach combined with human verification. By leveraging advanced technology to identify and normalize identical and similar products from competitors, you can optimize your pricing strategy and product assortment to remain competitive. Using Generative AI in normalizing color and size in fashion is key to powering competitive pricing intelligence at DataWeave.

    Continuous Adaptation and Learning

    Our solution evolves with your business, improving continuously through feedback and customization for retailers’ specific product categories. The system can be fine-tuned to understand and apply specialized tagging for niche or industry-specific product categories. This ensures that tags remain relevant and accurate across diverse catalogs and as trends emerge.

    The AI in our platform also continuously learns from user interactions and feedback, refining its tagging algorithms to improve accuracy over time.

    Stay Ahead of the Competition With Accurate Attribute Tagging

    In the current landscape, the ability to accurately and consistently tag product attributes is no longer a luxury—it’s essential for staying competitive. With advancements in Generative AI, companies like DataWeave are revolutionizing the way product tagging is handled, ensuring that every item in a retailer’s catalog is presented with precision and depth. As shoppers demand a more intuitive, seamless experience, next-generation tagging solutions are empowering businesses to meet these expectations head-on.

    DataWeave’s innovative approach to attribute tagging is more than just a technical improvement; it’s a strategic advantage in an increasingly competitive market. By leveraging AI to scale and automate tagging processes, online retailers can keep pace with expansive product assortments, manage content more effectively, and adapt swiftly to changes in consumer behavior. In doing so, they can maintain a competitive edge.

    To learn more, talk to us today!

  • Mastering Grocery Pricing Intelligence: A Strategic Approach for Modern Retailers

    Mastering Grocery Pricing Intelligence: A Strategic Approach for Modern Retailers

    When egg prices surged 70% during the 2023 avian flu outbreak, grocery retailers faced a critical dilemma: maintain margins and risk losing customers, or absorb costs and watch profits evaporate. Similarly, rising olive oil and chocolate prices also had domino effects, cascading down from retailers to consumers. In each of these scenarios, those with sophisticated pricing intelligence systems adapted swiftly, finding the sweet spot between competitiveness and profitability. Others weren’t so fortunate.

    This scenario continues to play out daily across thousands of products in the grocery sector. From breakfast cereals to fresh produce to bottled water, retailers must orchestrate pricing across a variety of categories – each with its own competitive dynamics, margin requirements, and price sensitivity patterns.

    The Evolution of Grocery Pricing Intelligence

    Imagine these scenarios in the grocery industry:

    • Milk prices spike during a supply shortage.
    • Your competitor drops egg prices by 20%.
    • Fresh produce costs fluctuate with an unseasonable frost.

    For grocery retailers, these aren’t occasional challenges—they’re Tuesday. Reacting to each pricing crisis as it comes isn’t just exhausting—it’s a recipe for shrinking margins and missed opportunities.

    Think of it this way: If you’re constantly playing defense with your pricing strategy, you’re already two steps behind. Commoditized items like milk and eggs face intense price competition, while seasonal products and fresh produce demand constant attention. Simply matching competitor prices or adjusting for cost changes isn’t enough anymore. What’s needed is a proactive approach that anticipates market shifts before they happen and turns pricing challenges into competitive advantages. This is where price management comes in.

    Price management has transformed from simple competitor checks into a strategic power play that can make or break a retailer’s market position. Weekly manual adjustments have given way to a long-term strategic view, driven by data analytics and market intelligence. Here are the basics of how price management in grocery retail works today.

    Three Pillars of Grocery Price Management

    1. Smart Data Collection: Building Your Foundation

    The journey begins with comprehensive data collection and storage across your entire product ecosystem. This means:

    • Complete Coverage Of All SKUs Across All Stores: Tracking prices for all SKUs across all stores, with particular attention to high-velocity items and volatile categories.
    • Dynamic Monitoring: Tracking prices across different time frequencies as grocery prices are highly volatile for different categories. So daily tracking for volatile items like dairy and produce, and weekly for more stable categories may be needed.
    • Competitive Intelligence: Gathering data not just on prices, but on promotions, pack sizes, and private label alternatives.
    • Infrastructure to Support Large Volumes of Data: Partnering with external data and analytics providers to bridge the gap when retailers struggle with the scale of digital infrastructure these data sets require.

    2. Intelligent Data Refinement: Making Sense of the Numbers

    Raw data alone isn’t enough—it needs context and structure to become actionable intelligence. This is called Data Refinement—the process of establishing meaningful relationships within the data to facilitate the extraction of valuable insights. This refinement stage is closely tied to the data collection strategy, as the quality and depth of the insights derived depend on the accuracy and coverage of the collected data.

    Data refinement includes several key processes:

    Advanced Product Matching

    Picture this: You’re tracking a competitor’s pricing on organic apples. Simple, right? Not quite. Yes, Universal Product Codes (UPCs) and Price Lookup Codes (PLUs) are present in Grocery to standardize product identification across different retailers—unlike the fashion industry’s endless style variations. Still, product matching isn’t as straightforward as scanning barcodes.

    Grocery Pricing Intelligence data faces a challenge when product names, weights, and details differ

    Here’s the catch: many retailer websites don’t display them. Then there’s the private label puzzle—your “Store’s Best” organic apples need to match against competitors’ house brands, each with their own unique UPC. Throw in different sizes (4 Apples vs. 1Kg of Apples), regional product names (fancy naming for plain old arugula), and international brand variations (like the name for Sprite in the USA and China), and you’ve got yourself a complex matching challenge that would make conventional pricing intelligence providers sweat.

    Grocery Pricing Intelligence data faces a challenge when different naming conventions and languages are used in different geographies

    Custom Product Relationships for Consistent Pricing and Competitive Positioning

    Think like a shopper browsing the dairy aisle. You regularly buy your family’s favorite organic yogurt, the 24oz tub. But today, you notice the larger 32oz size is on sale – except the 24oz isn’t. As you stand there, confused, you wonder: Is the sale only for the bigger size? Did I miss a promotion? Should I buy the 32oz even though it’s more than I need?

    For shoppers, this inconsistent pricing across product variations creates a frustrating experience. Establishing clear relationships between related items in your catalog is essential for maintaining consistent pricing and a coherent competitive strategy.

    Grocery Pricing Intelligence data refinement involves Custom Product Relationships for Consistent Pricing and Competitive Positioning

    Start by linking products based on attributes like size, brand, and packaging. That way, when you adjust the price of the 32oz yogurt, the 24oz version automatically updates too – no more scrambling to ensure uniform pricing across your assortment. Similarly, products of the same brand but with flavor variations should be connected to keep pricing consistent.

    Taking this one step further, mapping your competitors’ exact and similar products is crucial for comprehensive competitive intelligence. Distinguishing between premium and private label tiers, national brands, and regional players gives you a holistic view of the landscape. With this understanding, you can hone your pricing strategies to maintain a clear, compelling position across your entire category lineup.

    Consistent pricing, whether across your own product variations or against competitors, provides clarity and accuracy in your overall competitive positioning. By establishing these logical connections, you avoid the customer confusion of seemingly random, inconsistent discounts – and ensure your pricing strategies work in harmony, not disarray.

    The Role of AI and Data Sciences in Data Refinement

    On the surface, linking products based on attributes like size, brand, and packaging seems like a no-brainer. But developing and maintaining the systems to accurately and automatically identify these connections? That’s a whole different animal.

    Think about it – you’re not just dealing with text-based product titles and UPCs. There are images, videos, regional variations, private labels, and a whole host of other data types and industry nuances to account for.

    Luckily, DataWeave is one of the few companies that’s truly cracked the code. Our multimodal AI models are trained to process all those diverse data formats – from granular product specs to zany regional produce names. And it’s not just about technology; we also harness the power of human intelligence.

    See, in the grocery world, category managers are the real decision makers. They know their shelves inside and out and can spot those tricky connections in product matching, especially when they are not UPC-based. That’s why DataWeave built in a Human-in-the-Loop (HITL) process, where their AI systems continuously learn from expert feedback. It’s a feedback loop that allows our customers to pitch in and keep product relationships accurate, reliable, and always adapting to new market realities.

    So while product mapping may seem straightforward on the surface, the reality is it takes some serious horsepower to do it right. Thankfully, DataWeave has both the technical chops and the grocery industry know-how to make it happen. Because when it comes to pricing intelligence, getting those product connections right is half the battle.

    3. Strategic Implementation: Turning Insights into Action

    The true value of pricing intelligence (PI) is realized through its strategic application. Although many view PI as a technical function, its strategic significance is increasing, particularly in the context of recent economic pressures like inflation. Here’s why:

    Tactical vs Strategic Use of Data: From Standard Reporting to Competitive Analysis

    Pricing intelligence has come a long way from the days of simply reacting to daily price changes. These days, it’s not just about firefighting—it’s about driving long-term strategy.

    You can use pricing data to make quick, tactical adjustments, like matching a competitor’s sudden price drop on milk. Or, you can leverage that same data to predict market trends, optimize your product lineup, and shape your overall pricing strategy. Retailers who take that strategic view can get out ahead of the curve, anticipating shifts instead of just chasing them.

    DataWeave supports both of these approaches. Our Standard Reporting tools give pricing managers the nitty-gritty details they need—current practices, historical patterns, and operational KPIs. It’s all the insights you’d expect for making those tactical, day-to-day tweaks.

    In addition, DataWeave offers something more powerful: Competitive analysis. This is where pricing intelligence becomes a true strategic weapon. By providing a high-level view of market positioning, competitor moves, and untapped opportunities, competitive analysis empowers leadership to make proactive, big-picture decisions.

    Armed with this broader perspective, retailers can start taking a more surgical approach. Maybe you need to adjust pricing zones to better meet customer demands. Or rethink your overall strategies to stay ahead of the competition, not just keep pace. It’s the difference between constantly putting out fires and systematically fortifying your entire pricing fortress.

    Beyond Pricing: Comprehensive Data for Broader Insights

    Pricing intelligence is just the tip of the iceberg. When you really start to refine and harness your data, the possibilities for grocery retailers expand far beyond simple price comparisons. Think about it – all that information you’re collecting on products, markets, and consumer behavior? That’s a goldmine waiting to be tapped. Sure, you can use it to keep a pulse on competitor pricing. But why stop there?

    What if you could leverage that data to optimize your product assortment, making sure you’re stocking the right mix to meet customer demands? Or tap into predictive analytics to get a glimpse of future market shifts, so you can get out ahead of the curve? How about using it to streamline your supply chain, identify availability inefficiencies, and get products to shelves faster?

    Sure, pricing intelligence will always be mission-critical. But when you couple it with these other data-driven insights, that’s when grocery retailing gets really interesting. It’s about evolving from a price-matching robot to a true strategic visionary, armed with the intelligence to take your business to new heights.

    Looking Ahead: The Future of Grocery Pricing Intelligence

    The grocery pricing landscape continues to evolve, driven by:

    • Integration of AI and machine learning for predictive pricing
    • Enhanced focus on omnichannel pricing consistency
    • Growing importance of personalization in pricing strategies

    Pricing intelligence isn’t just about having data—it’s about having the right data and knowing how to use it strategically. Success requires a comprehensive approach that combines robust data collection, sophisticated analysis, and strategic implementation.

    By embracing modern pricing intelligence tools and strategies, grocery retailers can navigate market volatility, maintain competitive positioning, and drive sustainable growth. The key lies in building a pricing ecosystem that’s both sophisticated enough to handle complex data and flexible enough to adapt to changing market conditions.

    Ready to transform your pricing strategy? Check out our grocery price tracker to get month-on-month updates on grocery prices in the real world. Contact us to learn how our advanced pricing intelligence solutions can help your business stay ahead in the competitive grocery market.

  • Normalizing Size and Color in Fashion Using AI to Power Competitive Price Intelligence

    Normalizing Size and Color in Fashion Using AI to Power Competitive Price Intelligence

    Fashion is as dynamic a market as any—and more competitive than most others. Consumer trends and customer needs are always evolving, making it challenging for fashion and apparel brands to keep up.

    Despite the inherent difficulties fashion and apparel sellers face, this industry is one of the largest grossing markets in the world, estimated at $1.79 trillion in 2024. Global revenue for apparel is expected to grow at an annual rate of about 3.3% over the next four years. That means companies in this space stand to make significant revenue if they can competitively price their products, keep up with the competition, and win customer loyalty with consistent product availability.

    There are three main categories in fashion and apparel. These include:

    • Apparel and clothing (i.e., shirts, pants, dresses, and other apparel)
    • Footwear (i.e., sneakers, sandals, heels, and other products)
    • Accessories (i.e., bags, belts, watches, and so on)

    If you look at all of these product types across all sorts of retailers, there is a massive amount of overlapping data based on product attributes like style and size that are difficult to normalize.

    Fashion Attributes

    Style, color, and size are the main attribute categories in fashion and apparel. Style attributes include things like design, look, and overall aesthetics of the product. They’re very dependent on the actual product category of fashion as well. A shirt might have a slim fit attribute associated with it, whereas a belt might have a length. All these different attributes are usually labeled within a product listing and affect the consumer’s decision-making process:

    • Color (red, blue, sea green, etc.)
    • Pattern (solid, striped, checked, floral, etc.)
    • Material (cotton, polyester, leather, denim, silk, etc.)
    • Fit (regular, slim, relaxed, oversized, tailored, etc.)
    • Type (casual, formal, sporty, vintage, streetwear)

    Color Complexity in Fashion

    Color is perhaps the most visually distinctive attribute in fashion, yet it presents unique challenges for retailers. This is because color naming can vary across retailers and marketplaces. There are several major differences in color convention:

    • A single color can be labeled differently across brands (e.g., “navy,” “midnight blue,” “deep blue”)
    • Seasonal color names (e.g., “summer sage” vs. “forest green”)
    • Marketing-driven names (e.g., “sunset coral” vs. “pale orange”)
    Differences in color naming - challenges faced by fashion retail intelligence systems

    Size: The Other Critical Dimension

    Size in fashion refers to the dimensions or measurements that determine how fashion products fit. Depending on whether the product is a clothing item, shoes, or a hat, there will be different sizing options. Types of sizes include:

    • Standard sizes (XS, S, M, L, XL, XXL, XXL)
    • Custom sizes (based on brand, retailer, country, etc.)

    A single type of product may have different sizing labels. For instance, one pants listing may use traditional S, M, L, XL sizing, while another pants listing may use 24, 25, or 26, to refer to the waist measurement.

    Size Variations - challenges faced by fashion retail intelligence systems
    Size Variations - challenges faced by fashion retail intelligence systems
    Size Variations - challenges faced by fashion retail intelligence systems

    Size is a dynamic attribute that changes based on current trends. For example, there has recently been a significant shift towards inclusive sizing. Size inclusivity refers to the practice of selling apparel in a wide range of sizes to accommodate people of all body types. Consumers are more aware of this trend and are demanding a broader range of sizing offerings from the brands they shop from.

    In the US market, in particular, some 67% of American women wear a size 14 or above and may be interested in purchasing plus-size clothing. There is a growing demand in the plus-size market for more options and a wider selection. Many brands are considering expanding their sizes to accommodate more shoppers and tap into this growing revenue channel.

    Pricing Based on Size and Color

    Many fashion products are priced differently based on size and color. Let’s take a look at an example of what this can look like.

    Different colors may retail at different price points.

    A popular beauty brand (see image) is known for its viral lip tint. While most of the color variants are priced at $9.90 on Amazon, a specific colorway option, featuring less pigmented options, is priced at $9.57. This price differential is driven by both material costs and market demand.

    Different colorways (any of a range of combinations of colors in which a style or design is available) of the same product often command different prices also. This is based on:

    • Dye costs (some colors require more expensive processes)
    • Seasonal demand (traditional colors vs. trend colors)
    • Exclusivity (limited edition colors)

    An example of price variations by size is a women’s shirt that is being sold on Amazon as shown below. For this product, there are no style attributes to choose from. The only parameter the shopper has to select is the size they’d like to purchase. They can choose from S to XL. On the top, we can see that the product in size S is ₹389. Below, the size XL version of this same shirt is ₹399. This price increase is correlated to the change in size.

    Different sizes may retail at different price points.
    Different sizes may retail at different price points.

    So why are these same products priced differently? In an analysis of One Six, a plus-size clothing brand, several reasons for this difference in plus-size clothing were determined.

    • Extra material is needed, hence an increase in production costs
    • Extra stitching costs, hence an increase in production costs
    • Production of plus-size clothing often means acquiring specialized machinery
    • Smaller scale production runs for plus-size clothing means these initiatives often don’t benefit from cost savings

    Some sizes are sold more than others, meaning that in-demand sizes for certain apparel can affect pricing as well. Brands want to be able to charge as much as possible for their listing without risking losing a sale to a competitor.

    The Competitive Pricing Challenge: Normalizing Product Attributes Across Competitors in Apparel and Fashion

    There are hundreds of possible attribute permutations for every single apparel product. Some retailers may only sell core sizes and basic colors; some may sell a mix of sizes for multiple style types. Most retailers also sell multiple color variants for all styles they have on catalog. Other retailers may only sell a single, in-demand size of the product. Also, when other retailers are selling the product, it’s unlikely that their naming conventions, color options, style options, and sizing match yours one-for-one.

    In one analysis, it was found that there were 800+ unique values for heel sizes and 1000+ unique values for shirts and tops at a single retailer! If you’re looking to compare prices, the effort involved in setting up and managing lookup tables to identify discrepancies when one retailer uses European sizes and another uses USA sizes, for example, is simply too onerous to contemplate doing. Colors only add to the complexity – as similar colors may have new names in different regions and locations as well!

    Even if you managed to find all the discrepancies between product attributes, you would still need to update them any time a competitor changed a convention.

    Still, monitoring your competitors and strategically pricing your listings is essential to maintain and grow market share. So what do you do? You can’t simply eyeball your competitor’s website to check their pricing and naming conventions. Instead, you need advanced algorithms to scan the entire marketplace, identify individual products being sold, and normalize their data and attributes for analysis.

    Getting Color and Size Level Pricing Intelligence

    With DataWeave, size and color are just two of several dimensions of a product instead of an impossible big data problem for teams. Our product matching engine can easily handle color and sizing complexity via our AI-driven approach combined with human verification.

    This works by using AI built on more than 10 years of product catalog data across thousands of retail websites. It matches common identifiers, like UPC, SKU code, and other attributes for harmonization before employing a large language model (LLM) prompts to normalize color variations and sizing to a single standard.

    The data flow DataWeave uses for product sizing and color normalization

    For example, if a competitor has the smallest size listed as Sm but has your smallest listing identified as S, DataWeave can match those two attributes using AI. Similar classification can be performed on color as well.

    Complex LLM prompts are pre-established so that this process is fast and efficient, taking minutes rather than weeks of manual effort.

    Harmonizing products along with their color and sizing data across different retailers for further analysis has several benefits. Most importantly, product matching helps teams conduct better competitive analysis, allowing them to stay informed about market trends, competitors’ offerings, and how those competitors are pricing various permutations of the same product. It helps ensure that you’re offering the most competitive assortment of sizing in several colors to win more market share as well. Overall, it’s easier for teams to gain insights and exploit their findings when all the data is clean and available at their fingertips.

    Product Matching Size and Color in Apparel and Fashion

    Color and size are crucial attributes for retailers and brands in the apparel and fashion industry. It adds a level of complexity that can’t be overstated. While it’s a necessity to win consumers (more colors and sizes will mean a wider potential reach), the more permutations you add to your listing, the more complicated it will be to track it against your competition. However, This challenge is worth undertaking as long as you have the right solutions at your disposal.

    With a strategy backed by advanced technology to discover identical and similar products across the competitive landscape and normalize their color and sizing attributes, you can ensure that you are competitively pricing your products and offering the best assortment possible. Employing DataWeave’s AI technology to find competitor listings, match products across variants, and track pricing regularly is the way to go.

    Interested in learning more about DataWeave? Click here to get in touch!

  • DataWeave’s AI Evolution: Delivering Greater Value Faster in the Age of AI and LLMs

    DataWeave’s AI Evolution: Delivering Greater Value Faster in the Age of AI and LLMs

    In retail, competition is fierce, and in its ever-evolving landscape, consumer expectations are higher than ever.

    For years, our AI-driven solutions have been the foundation that empowers businesses to sharpen their competitive pricing and optimize digital shelf performance. But in today’s world, evolution is constant—so is innovation. We now find ourselves at the frontier of a new era in AI. With the dawn of Generative AI and the rise of Large Language Models (LLMs), the possibilities for eCommerce companies are expanding at an unprecedented pace.

    These technologies aren’t just a step forward; they’re a leap—propelling our capabilities to new heights. The insights are deeper, the recommendations more precise, and the competitive and market intelligence we provide is sharper than ever. This synergy between our legacy of AI expertise and the advancements of today positions DataWeave to deliver even greater value, thus helping businesses thrive in a fast-paced, data-driven world.

    This article marks the beginning of a series where we will take you through these transformative AI capabilities, each designed to give retailers and brands a competitive edge.

    In this first piece, we’ll offer a snapshot of how DataWeave aggregates and analyzes billions of publicly available data points to help businesses stay agile, informed, and ahead of the curve. These fall into four broad categories:

    • Product Matching
    • Attribute Tagging
    • Content Analysis
    • Promo Banner Analysis
    • Other Specialized Use Cases

    Product Matching

    Dynamic pricing is an indispensable tool for eCommerce stores to remain competitive. A blessing—and a curse—of online shopping is that users can compare prices of similar products in a few clicks, with most shoppers gravitating toward the lowest price. Consequently, retailers can lose sales over minor discrepancies of $1–2 or even less.

    All major eCommerce platforms compare product prices—especially their top selling products—across competing players and adjust prices to match or undercut competitors. A typical product undergoes 20.4 price changes annually, or roughly once every 18 days. Amazon takes it to the extreme, changing prices approximately every 10 minutes. It helps them maintain a healthy price perception among their consumers.

    However, accurate product matching at scale is a prerequisite for the above, and that poses significant challenges. There is no standardized approach to product cataloging, so even identical products bear different product titles, descriptions, and attributes. Information is often incomplete, noisy, or ambiguous. Image data contains even more variability—the same product can be styled using different backgrounds, lighting, orientations, and quality; images can have multiple overlapping objects of interest or extraneous objects, and at times the images and the text on a single page might belong to completely different products!

    DataWeave leverages advanced technologies, including computer vision, natural language processing (NLP), and deep learning, to achieve highly accurate product matching. Our pricing intelligence solution accurately matches products across hundreds of websites and automatically tracks competitor pricing data.

    Here’s how it works:

    Text Preprocessing

    It identifies relevant text features essential for accurate comparison.

    • Metadata Parsing: Extracts product titles, descriptions, attributes (e.g., color, size), and other structured data elements from Product Description Pages (PDP) that can help in accurately identifying and classifying products.
    • Attribute-Value Normalization: Normalize attributes names (e.g. RAM vs Memory) and their values (e.g., 16 giga bytes vs 16 gigs vs 16 GB); brand names (e.g., Benetton vs UCB vs United Colors of Benetton); mapping category hierarchies a standard taxonomy.
    • Noise Removal: Removes stop words and other elements with no descriptive value; this focuses keyword extraction on meaningful terms that contribute to product identification.

    Image Preprocessing

    Image processing algorithms use feature extraction to define visual attributes. For example, when comparing images of a red T-shirt, the algorithm might extract features such as “crew neck,” “red,” or “striped.”

    Image Preprocessing using advanced AI and other tech for product matching in retail analytics.

    Image hashing techniques create a unique representation (or “hash”) of an image, allowing for efficient comparison and matching of product images. This process transforms an image into a concise string or sequence of numbers that captures its essential features even if the image has been resized, rotated, or edited.

    Before we perform these activities there is a need to preprocess images to prepare them for downstream operations. These include object detection to identify objects of interest, background removal, face/skin detection and removal, pose estimation and correction, and so forth.

    Embeddings

    We have built a hybrid or a multimodal product-matching engine that uses image features, text features, and domain heuristics. For every product we process we create and store multiple text and image embeddings in a vector database. These include a combination of basic feature vectors (e.g. tf-idf based, colour histograms, share vectors) to more advanced deep learning algorithms-based embeddings (e.g., BERT, CLIP) to the latest LLM-based embeddings.

    Classification

    Classification algorithms enhance product attribute tagging by designating match types. For example, the product might be identified as an “exact match”, “variant”, “similar”, or “substitute.” The algorithm can also identify identical product combinations or “baskets” of items typically purchased together.

    What is the Business Impact of Product Matching?

    • Pricing Intelligence: Businesses can strategically adjust pricing to remain competitive while maintaining profitability. High-accuracy price comparisons help businesses analyze their competitive price position, identify opportunities to improve pricing, and reclaim market share from competitors.
    • Similarity-Based Matching: Products are matched based on a range of similarity features, such as product type, color, price range, specific features, etc., leading to more accurate matches.
    • Counterfeit Detection: Businesses can identify counterfeit or unauthorized versions of branded products by comparing them against authentic product listings. This helps safeguard brand identity and enables brands to take legal action against counterfeiters.

    Attribute Tagging

    Attribute tagging involves assigning standardized tags for product attributes, such as brand, model, size, color, or material. These naming conventions form the basis for accurate product matching. Tagging detailed attributes, such as specifications, features, and dimensions, helps match products that meet similar criteria. For example, tags like “collar” or “pockets” for apparel ensure high-fidelity product matches for hard-to-distinguish items with minor stylistic variations.

    Attributes that are tagged when images are matched for retail ecommerce analytcis.

    Including tags for synonyms, variants, and long-tail keywords (e.g., “denim” and “jeans”) improves the matching process by recognizing different terms used for similar products. Metadata tags categorize similar items according to SKU numbers, manufacturer details, and other identifiers.

    Altogether, these capabilities provide high-quality product matches and valuable metadata for retailers to classify their products and compare their product assortment to competitors.

    User-Generated Content (UGC) Analysis

    Customer reviews and ratings are rich sources of information, enabling brands to gauge consumer sentiment and identify shortcomings regarding product quality or service delivery. However, while informative, reviews constitute unstructured “noisy” data that is actionable only if parsed correctly.

    Here’s where DataWeave’s UGC analysis capability steps in.

    • Feature Extractor: Automatically pulls specific product attributes mentioned in the review (e.g., “battery life,” “design” and “comfort”)
    • Feature Opinion Pair: Pairs each product attribute with a corresponding sentiment from the review (e.g., “battery life” is “excellent,” “design” is “modern,” and “comfort” is “poor”)
    • Calculate Sentiment: Calculates an overall sentiment score for each product attribute
    The user generated content analysis framework used by DataWeave to calculate sentiment.

    The final output combines the information extracted from each of these features, which looks something like this:

    • Battery life is excellent
    • Design is modern
    • Not satisfied with the comfort

    The algorithm also recognizes spammy reviews and distinguishes subjective reviews (i.e., those fueled by emotion) from objective ones.

    DataWeave's image processing tool also analyses promo banners.

    Promo Banner Analysis

    Our image processing tool can interpret promotional banners and extract information regarding product highlights, discounts, and special offers. This provides insights into pricing strategies and promotional tactics used by other online stores.

    For example, if a competitor offers a 20% discount on a popular product, you can match or exceed this discount to attract more customers.

    The banner reader identifies successful promotional trends and patterns from competitors, such as the timing of discounts, frequently promoted product categories or brands, and the duration of sales events. Ecommerce stores can use this information to optimize their promotion strategies, ensuring they launch compelling and timely offers.

    Other Specialized Use Cases

    While these generalized AI tools are highly useful in various industries, we’ve created other category—and attribute-specific capabilities for specialty goods (e.g., those requiring certifications or approval by federal agencies) and food items. These use cases help our customers adhere to compliance requirements.

    Certification Mark Detector

    This detector lets retailers match items based on official certification marks. These marks represent compliance with industry standards, safety regulations, and quality benchmarks.

    Example:

    • USDA Organic: Certification for organic food production and handling
    • ISO 9001: Quality Management System Certification

    By detecting these certification marks, the system can accurately match products with their certified counterparts. By identifying which competitor products are certified, retailers can identify products that may benefit from certification.

    Image analysis based product matching at DataWeave also detects certificate marks.

    Nutrition Fact Table Reader

    Product attributes alone are insufficient for comparing food items. Differences in nutrition content can influence product category (e.g., “health food” versus regular food items), price point, and consumer choice. DataWeave’s nutrition fact table reader scans nutrition information on packaging, capturing details such as calorie count, macronutrient distribution (proteins, fats, carbohydrates), vitamins, and minerals.

    The solution ensures items with similar nutritional profiles are correctly identified and grouped based on specific dietary requirements or preferences. This helps with price comparisons and enables eCommerce stores to maintain a reliable database of product information and build trust among health-conscious consumers.

    Image processing for product matching also extracts nutrition table data at DataWeave.

    Building Next-Generation Competitive and Market Intelligence

    Moving forward, breakthroughs in generative AI and LLMs have fueled substantial innovation, which has enabled us to introduce powerful new capabilities for our customers.

    How Gen AI and LLMs are used by DataWeave to glean insights for analytics

    These include:

    • Building Enhanced Products, Solutions, and Capabilities: Generative AI and LLMs can significantly elevate the performance of existing solutions by improving the accuracy, relevance, and depth of insights. By leveraging these advanced AI technologies, DataWeave can enhance its product offerings, such as pricing intelligence, product matching, and sentiment analysis. These tools will become more intuitive, allowing for real-time updates and deeper contextual understanding. Additionally, AI can help create entirely new solutions tailored to specific use cases, such as automating competitive analysis or identifying emerging market trends. This positions DataWeave to remain at the forefront of innovation, offering cutting-edge solutions that meet the evolving needs of retailers and brands.
    • Reducing Turnaround Time (TAT) to Go-to-Market Faster: Generative AI and LLMs streamline data processing and analysis workflows, enabling faster decision-making. By automating tasks like data aggregation, sentiment analysis, and report generation, AI dramatically reduces the time required to derive actionable insights. This efficiency means that businesses can respond to market changes more swiftly, adjusting pricing or promotional strategies in near real-time. Faster insights translate into reduced turnaround times for product development, testing, and launch cycles, allowing DataWeave to bring new solutions to market quickly and give clients a competitive advantage.
    • Improving Data Quality to Achieve Higher Performance Metrics: AI-driven technologies are exceptionally skilled at cleaning, organizing, and structuring large datasets. Generative AI and LLMs can refine the data input process, reducing errors and ensuring more accurate, high-quality data across all touchpoints. Improved data quality enhances the precision of insights drawn from it, leading to higher performance metrics like better product matching, more accurate price comparisons, and more effective consumer sentiment analysis. With higher-quality data, businesses can make smarter, more informed decisions, resulting in improved revenue, market share, and customer satisfaction.
    • Augmenting Human Bandwidth with AI to Enhance Productivity: Generative AI and LLMs serve as powerful tools that augment human capabilities by automating routine, time-consuming tasks such as data entry, classification, and preliminary analysis. This allows human teams to focus on more strategic, high-value activities like interpreting insights, building relationships with clients, and developing new business strategies. By offloading these repetitive tasks to AI, human productivity is significantly enhanced. Employees can achieve more in less time, increasing overall efficiency and enabling teams to scale their operations without needing a proportional increase in human resources.

    In our ongoing series, we will dive deep into each of these capabilities, exploring how DataWeave leverages cutting-edge AI technologies like Generative AI and LLMs to solve complex challenges for retailers and brands.

    In the meantime, talk to us to learn more!

  • Competitor Price Monitoring in E-commerce: Everything You Need to Know

    Competitor Price Monitoring in E-commerce: Everything You Need to Know

    Picture this: You wake up one morning to discover that your top competitor reduced their prices overnight. And now your shopper traffic has tanked and your sales have taken a hit.

    Unfortunately, this is a common scenario because your customers can compare prices online in seconds—and loyalty lies with the budget.

    So, how can you protect your business? Price monitoring.

    Price monitoring solutions can help you keep abreast of competitor price changes—which, of course, will help you improve your pricing strategies, retain your customers, and maximize your profits.

    How? In this article, we’ll explore:

    • What is price monitoring
    • The key benefits of price monitoring for retailers and brands
    • What a capable price monitoring solution can do

    What Is Price Monitoring?

    Price monitoring is the process of tracking and analyzing your competitor’s prices across various online and offline platforms. By monitoring competitors’ prices, you can understand market price trends and adjust your prices strategically—which, in turn, helps you remain competitive, increase margins, and improve customer retention.

    5 Benefits of Price Monitoring

    Competitor price monitoring can help you:

    1. Gain a competitive edge: Competitor price tracking allows you to adjust your prices to remain attractive to consumers.
    2. Maximize revenue: With timely pricing data, you’re empowered to identify optimum price points that strike a delicate balance between maximizing revenue and maintaining customer loyalty.
    3. Retain customers: Consumers are looking for the most value for their dollar, so maintaining consistently competitive pricing is crucial for retaining loyal customers.
    4. Understand promotional effectiveness: Price monitoring helps businesses evaluate the effectiveness of their promotions and discounts. By comparing the impact of different pricing strategies, businesses can refine their promotional tactics to maximize sales and customer engagement.
    5. Understand market movements: By analyzing historical pricing data, you’re better positioned to anticipate future pricing changes — and adjust your strategies accordingly.

    4 Essential Capabilities of Price Monitoring Software

    Here are four capabilities to look for when choosing a price monitoring system.

    1. AI-Driven Product Matching

    Product matching is the process of identifying identical or similar products across different platforms to ensure accurate price comparisons.

    If your price monitoring solution can’t reliably match your products with competitors’ across various sales channels at scale, you’ll end up with poor data. Inaccurate data will then lead you to make misinformed pricing decisions.

    Product matching needs to be accurate and comprehensive, covering a wide range of products and product variations—even for including private label products.

    For example, AI-driven product matching can recognize a specific brand and model of sneakers across multiple online stores—even if product descriptions and images differ. Here’s how it works in a nutshell:

    • Sophisticated algorithms and deep learning architecture enable AI to identify and match products that aren’t identical but share key characteristics and features.
    • Using unified systems for text and image recognition, the AI matches similar SKUs across hundreds of eCommerce stores and millions of products.
      The AI zeroes in on critical product elements in images, like a t-shirt’s shape, sleeve length, and color.
    • The AI also extracts unique signatures from photos for rapid, efficient identification and grouping across billions of indexed items.

    DataWeave’s AI algorithm can initially match products with 80–90% accuracy. Then, humans can bring contextual judgement and make nuanced decisions that the AI might miss to correct errors quickly and push for accuracy closer to 100%. By integrating AI automation with human validation, you can achieve accurate and reliable product-matching coverage at scale.

    2. Accurate and Comprehensive Data Collection and Aggregation

    The insights you derive are only as good as the data you collect. However, capturing comprehensive pricing data is tough when your competitors operate on multiple platforms.

    For truly effective price monitoring insights, you need consistent, comprehensive, and highly accurate data. This means your chosen price monitoring system should:

    • Scrape data from various sources, such as desktop and mobile sites and mobile applications.
    • Pull data from various online platforms like aggregators, omnichannel retailers, delivery intermediaries, online marketplaces, and more.
    • Handle data from different regions and languages.
    • Collect data at regular intervals to ensure timeliness.

    DataWeave’s online price monitoring software covers all of these bases and more with a fast, automated data source configuration system. It also allows you to painlessly add new data sources to scrape.

    Instead of incomplete or inaccurate data, you’ll have comprehensive and up-to-date data, allowing you to respond quickly to market changes with confidence.

    3. Seamless Normalization of Product Measurement Units

    You can’t compare apples to oranges—or price-per-kilogram to price-per-pound.

    For price monitoring to be accurate, there must be a way to normalize measurement units—so that we’re always comparing price-per-gram to price-per-gram. If we compare prices without taking into account measurement units, our data will be misleading at best.

    Let’s take a closer look. Say that your top competitor sells 12oz cans of beans for $3, and you sell 15oz cans for $3.20. At first glance, your larger cans of beans will appear more expensive—but that’s not true. If we normalize the measurement unit—in this example, an oz—the larger can of beans offers more value to customers.

    Unit of measure normalization facilitates sound price adjustments based on accurate and reliable data. For this reason, every business needs a price tracking tool that can guarantee accurate comparisons by normalizing unit measurements—including weight, volume, and quantity.

    4. Actionable Data and an Intuitive User Experience

    Knowledge is only powerful when applied—and price monitoring insights are only useful when they’re accessible and actionable.

    For this reason, the best price monitoring software doesn’t just provide insights based on accurate and comprehensive data, but it also provides several ways to understand and deploy those insights.

    Ideal price monitoring solutions provide customized pricing alerts, intuitive dashboards, detailed reports, and visuals that are easy to interpret—all tailored to each particular team or a team member’s needs. These features should make it easy for team members to compare prices against those of competitors in specific categories and product groupings.

    Your price tracking tool should also permit flexible API integrations and offer straightforward data export options. This way, you can integrate competitive pricing data with your pricing software, Business Intelligence (BI) tools, or Enterprise Resource Planning (ERP) system.

    4 Ways Retailers Can Leverage Price Monitoring

    Retailers can use price monitoring tools to remain competitive without compromising profitability—here’s how:

    1. Track Competitors’ Prices

    Competitor price monitoring helps you avoid being undercut—and, as a result, maintain market share. By tracking competitor prices in real-time, you can adjust prices to remain competitive, especially in dynamic markets. Ideally, you should monitor both direct competitors selling the same products and indirect competitors selling similar or alternative products. This way, you’ll have a complete picture of market prices and can make more informed pricing adjustments.

    2. Understand Historical and Seasonal Price Trends

    As a retailer, you may want to analyze historical data to identify price patterns and predict future price movements—especially in relation to holidays and seasonal products. Knowing what’s coming, you’re better positioned to plan for pricing changes and promotional campaigns.

    3. Implement Dynamic Pricing

    Dynamic pricing is the process of adjusting prices based on real-time market conditions, product demand, and competitors’ prices—allowing you to respond faster to market changes to maintain optimized prices.

    4. Optimize Promotional Strategies

    Price monitoring tools can track retail promotions across numerous online and offline sales avenues, providing insight into the nature and timing of competitors’ promotions. This data can help you determine which promotions are most effective—and which aren’t—allowing you to improve your own promotions and discounts, and allocate marketing resources where it matters most. This is especially beneficial during peak sales periods.

    3 Ways Brands Can Employ Price Monitoring

    Here are three ways brands can use price monitoring to remain profitable, protect brand equity, and gain a competitive edge.

    1. Maintain Consistent Retail Prices

    Minimum advertised price (MAP) policies are designed to prevent retailers from devaluing a brand while ensuring fair competition among retailers. Price monitoring applications allow your brand to track retailers’ prices to detect MAP policy violations. Data in hand, you can maintain consistent pricing across online sales channels, physical stores, and retail stores’ digital shelves — and, critically, protect your brand equity.

    2. Improve Product and Brand Positioning

    When you understand how your products’ prices compare to those of competitors, you can set prices to improve brand positioning. For example, if you want to position your brand as luxurious and high-quality, you need to set higher product prices than budget-friendly alternative products.

    3. Ensure Product Availability

    You can use a price monitoring solution to track product availability to ensure products are always in stock, even across different physical stores and online marketplaces. If a product is frequently sold out, you can adjust production levels or help retailers to improve their inventory management.

    Key Takeaways: E-commerce Price Monitoring

    Price monitoring software allows you to compare your products’ prices with competitors. This valuable data can help you:

    • Optimize revenue through timely price changes and dynamic pricing
      Avoid being undercut by competitors
    • Improve pricing strategies and promotions to increase sales and retain customers
    • Maintain consistent prices across sales channels

    To learn more, check out our article, What is Competitive Pricing Intelligence: The Ultimate Guide here or reach out and talk to us today!

  • Cracking the Code: How Retailers Can Adapt to Plummeting Egg Prices in 2024

    Cracking the Code: How Retailers Can Adapt to Plummeting Egg Prices in 2024

    Virtually every cuisine in the world uses eggs. They’re in your breakfast, lunch, dinner, and dessert — which is perhaps why the global egg market is expected to generate $130.70 billion in revenue in 2024 and is projected to grow to approximately $193.56 billion by 2029.

    More specifically, the United States is the fourth-largest egg producer worldwide. The country’s egg market is projected to generate $15.75 billion in 2024 and increase to $22.51 billion by 2029.

    This growth is driven by several factors, most notably:

    • Health-consciousness among consumers: Consumers value eggs for their essential nutrients and rich protein content.
    • Demand for convenience foods: Consumers’ preferences are shifting toward quick and easy foods, which drives demand for shell eggs and pre-packaged boiled or scrambled eggs.
    • Population Growth: A growing worldwide population increases the demand for eggs.
    • Affordability and accessibility: Eggs are an affordable and accessible nutrient-dense food source for many.

    Despite these factors contributing to the U.S. egg market’s growth, recent times have seen egg prices fall dramatically.

    Based on a sample of 450 SKUs, DataWeave discovered that egg prices in the U.S. fell by 6.7% between April 2023 and April 2024, dipping to its lowest (-12.6%) in December 2023.

    Egg Price Chart: Egg Prices USA Going Down 98.95% between April 2023 and April 2024

    So, what’s causing the decrease in egg prices?

    The Rise and Fall of Egg Prices: A Recent History

    In 2022, avian influenza severely impacted the United States. The disease affected wild birds in nearly every state and devastated commercial flocks in approximately half of the country.

    The 2022 incident was the first major outbreak since 2015 and led to the culling of more than 52.6 million birds, mainly poultry, to prevent the disease from spreading uncontrollably.

    With almost 12 million fewer egg-laying hens, the United States produced around 109.5 billion eggs in 2022 — a drop of nearly two billion from the previous year.

    Consequently, the cost of eggs soared, peaking at $4.82 a dozen — more than double the price of eggs in the previous year.

    The avian flu continues to affect egg-laying hens and other poultry birds across the United States. As of April 2024, farms have killed a total of 85 million poultry birds in an attempt to contain the disease.

    Despite the disease’s effects, production facilities have made significant efforts to repopulate flocks, leading to a steady increase in supply – and a much anticipated decrease in egg prices.

    According to the U.S. Bureau of Labor Statistics, there was an increase in producer egg prices in 2022, reaching a peak in November 2022, at which point they began to fall.

    Retailer’s egg prices followed suit. The egg price chart below depicts retailers’ declining egg prices over one year, from April 2023 to April 2024, with Giant Eagle showing the most significant price reductions and Walmart the least.

    Egg Price Chart Featuring Leading Retailers 2023-2024

    What Does the Future Hold for Egg Prices?

    The USDA reported recent severe avian flu outbreaks in June 2024. These outbreaks are estimated to have affected 6.23 million birds.

    With a reduction in egg-laying hens, egg prices are likely to increase — time will tell.

    Nonetheless, the annual per capita consumption of eggs in the U.S. is projected to reach 284.4 per person in 2024 from 281.3 per person in 2023. So for now, producers and retailers can rest assured of the growing demand for eggs.

    How Can Retailers Adapt to the Unpredictability of Egg Prices?

    Egg prices were down to $2.69 for a dozen in May 2024. However, they are still significantly higher than consumers were used to just a few years ago—eggs were, on average, $1.46 a dozen in early 2020.

    Additionally, while the avian flu puts pressure on producers, inflation and supply chain disruptions exert pressure on retailers.

    With such challenging egg market conditions, what can retailers do to maintain customer loyalty amid reduced consumer spending while maintaining profitability?

    1. Give the Customer What They Want: Increase Offerings of Organic, Cage-Free, and Free-Range Eggs

    As mentioned, Data Bridge Market Research’s trends and forecast report highlighted a significant increase in consumer health consciousness. Additionally, animal welfare increasingly influences consumers’ purchasing decisions when buying meat and dairy products.

    DataWeave data shows that the prices of organic, cage-free, and free-range eggs—such as those by brands like Happy Eggs and Marketside—have fallen less than those of non-organic, caged egg brands.

    Egg Price Chart Featuring Leading Egg Brand Prices 2023-2024

    2. Increase Private-Label Offerings

    Private labels typically offer retailers higher margins than national brands. These margins can shield consumers from sudden wholesale egg price swings, helping to preserve brand trust and consumer loyalty without sacrificing profitability.

    Moreover, eggs are particularly suited to private labeling, given their uniform appearance and taste and the lack of product innovation opportunities.

    Undoubtedly, this is why sales of private-label eggs dwarf sales of national egg brands in the United States. Statista reports that across three months in 2024, private label egg sales amounted to $1.55 billion U.S. dollars, while the combined sales of the top nine national egg brands totaled just $617.88 million U.S. dollars.

    3. Price Intelligently

    With the current and predicted fluctuations in egg prices over the foreseeable future, price competitiveness is paramount to margin management and customer loyalty.

    This is especially true when lower prices are the primary factor influencing the average consumer’s choice of supermarket for daily essentials purchases.

    AI-driven pricing intelligence tools like DataWeave give retailers valuable highly granular and reliable insights on competitor pricing and market dynamics. In today’s data-motivated environment, these insights are necessary for competitiveness and profitability.

    Final Thoughts

    Egg prices have fluctuated significantly due to the impact of avian flu. Despite recent price drops, future egg price increases are possible due to ongoing outbreaks. Retailers should adapt to unstable egg prices by increasing organic, free-range, cage-free, and private-label egg offerings while leveraging AI-driven pricing tools to maintain margins and customer loyalty.

    Speak to us today to learn more!

  • How Healthy is Your Assortment?

    How Healthy is Your Assortment?

    In 2025, both consumers and retailers continue to prioritize better health – albeit with evolving definitions and expectations.

    The pandemic fundamentally transformed how consumers approach wellness, with this shift becoming entrenched in shopping behaviors years later. As shopping habits have permanently altered, retailers now face increased pressure to rapidly adapt their assortments with in-demand health and wellness products that enhance customer experience across various channels – online and offline.

    Let’s explore how leading retailers are keeping consumers – and their own bottom lines – healthy by responding effectively to market trends to drive online sales and market share.

    Health & Wellness Influence The Product Mix Across Categories

    Consumption habits have changed dramatically since the onset of the pandemic. A McKinsey study shows that 82% and 73% of US, and UK consumers respectively now consider health & wellness a top priority. Typically shoppers adjust grocery shopping and meal planning at the start of the year, with many focusing on fresh, organic, and nutrient-rich foods.

    The influential health and wellness mega-trend spans diverse retail channels, including grocery, pharmacy and mass. It extends across numerous categories like:

    • Food and beverage (natural, organic, vegan, plant-based food)
    • Health and personal care
    • Beauty
    • Cleaning products
    • Fitness equipment 
    • Athleisure (apparel)
    • Consumer electronics like health wearables.

    Today’s health movement is so powerful and compelling that retailers have revised their business strategies to better serve health-conscious consumers. For instance, drugstores are reinventing themselves as healthcare destinations, with CVS and Kroger expanding into personalized care delivery and value-based clinics to enhance their health offerings.

    Major retailers like Amazon, Walmart, and Target report robust sales in health and wellness categories. For example, Walmart saw a 4.6% increase in comparable sales in early 2024, driven significantly by grocery, consumables, and health-related products.

    New product categories are gaining traction:

    • Functional foods and beverages are seeing unprecedented growth, with Target launching over 2,000 wellness items in the category, including exclusive products priced under $10.
    • Personalized nutrition and mental health products are surging, including tailored dietary solutions and stress-reducing items.
    • Health wearables and wellness tech continue to rise in popularity, with over 150 new wellness tech items launched at Target this year, including innovative red-light therapy devices.
    • Transparency and sustainability certifications like organic, non-GMO, and vegan labels are increasingly driving purchasing decisions.
    • Clinically proven benefits offered by health & wellness products are gaining traction among Gen Z.

    Retail’s Survival Of The Fittest Moves Online

    As the omnichannel retail sector continues to grow, more shoppers now make purchase decisions within minutes using just a few clicks rather than physically visiting brick-and-mortar stores. In some cases, AI agents like Operator from Chat-GPT or Gemini (Google’s Chatbot) even make personalized, curated lists and reduce the time taken to make purchase decisions. Traditional retail paradigms are rapidly becoming obsolete as consumers grow savvier, more empowered, and better informed than ever before.

    To stay competitive, more retailers are embracing AI-driven data insights to adjust their assortments to reflect consumer demand for health and wellness products.

    According to industry experts, data insights have emerged as a critical retail strategy that continues to gain momentum. This is because retailers can no longer afford to guess how to approach their omnichannel strategy. They need the accuracy, clarity, and efficiency of data insights to guide their assortment and pricing decisions to outmaneuver competitors, maximize sales, and win market share as shopping evolves online.

    Among its retail best practices, Bain & Company recommends retailers “lead with superior assortments that use a customer-centric lens to reduce complexity and increase space for the products customers love.” Insights can help retailers discover the optimal mix of national brands, private labels, limited-time offers, and value-added bundles.

    Lead with superior assortments …
    increase space for the products consumers love

    ~ Bain & Company

    Determining the optimal mix of products also includes bestsellers and unique items that help retailers distinguish their offerings. Assortment insights help retail executives track competitors’ assortment changes and spot gaps in their own product assortment to adapt to emerging consumer trends and in-demand products.

    Why Effective Assortment Planning Matters

    Assortment planning sits at the heart of retail success, directly influencing profitability, customer satisfaction, and competitive differentiation. In today’s health-conscious market, getting your assortment right means:

    • Meeting Customer Expectations: Today’s health-conscious consumers expect relevant, high-quality products that match their wellness goals. A well-planned assortment signals that a retailer understands its customers’ evolving needs.
    • Optimizing Inventory Investment: Strategic assortment planning ensures capital is allocated to products with the highest return potential while minimizing investments in slow-moving items.
    • Creating Competitive Advantage: A distinctive assortment that includes popular health and wellness products alongside unique offerings helps retailers stand out in a crowded marketplace.
    • Reducing Lost Sales: Effective assortment planning minimizes the risk of stockouts on high-demand health and wellness items, preventing customers from shopping elsewhere.
    • Supporting Omnichannel Strategies: Well-executed assortment planning ensures consistency across physical and digital touchpoints, creating a seamless customer experience.
    • Improving Operational Efficiency: A thoughtfully curated assortment reduces complexity throughout the supply chain, from procurement to warehouse management to in-store operations.

    As health and wellness continues to drive consumer spending, retailers who excel at assortment planning can capitalize on these trends more effectively than their competitors, turning market insights into tangible business results.

    AI-Powered Assortment Analytics Driving Retail Success

    The synergy of AI and data analytics into retail assortment planning is changing how businesses approach inventory management. Retailers using AI-driven predictive analytics have achieved a 36% SKU reduction while increasing sales by 1-2%, showcasing the efficiency of data-driven approaches according to a McKinsey report.

    Retailers face several challenges that can hinder strategic assortment planning:

    • Limited Understanding of Competition: Retailers struggle to gain comprehensive insights into their product assortments relative to competitors, often lacking visibility into their strengths and weaknesses across categories.
    • Data Overload: Assortment planning involves handling vast amounts of data, making it challenging for category managers to extract actionable insights without user-friendly tools and visualization.
    • Cross-Channel Consistency: With omnichannel retailing, ensuring consistency across physical stores, e-commerce, and other channels is complex. Misalignment can lead to customer dissatisfaction and loss of loyalty.
    • Adapting to Changing Market Trends: Identifying top-selling products and tracking consumer preferences is challenging. Balancing the right mix of products is crucial; without analytics, retailers risk lost sales or excess slow-moving inventory.
    • Scalability and Efficiency: As retailers expand into new markets or categories, scaling their assortment planning processes efficiently becomes a challenge. Legacy systems and manual methods often fail to support the agility needed for quick decision-making at scale.

    DataWeave’s Assortment Analytics helps retailers address these challenges by providing a robust, easy-to-use platform that delivers actionable insights into product assortments and competitive positioning. With AI-driven, contextual insights and alerts, retailers can effortlessly identify high-demand, unique products, capitalize on catalog strengths, optimize pricing and promotions, improve stock availability, and refine assortments to maintain a competitive edge.

    Beyond Data: Actionable Insights That Drive Results

    DataWeave’s platform provides a comprehensive, insight-led view into assortments through several key dimensions:

    • Stock Insights: Monitor stock changes across retailers to stay updated on availability.
    • Category and Sub-Category Insights: Analyze assortment changes, identify newly introduced or discontinued categories, and track leading retailers in specific segments.
    • Brand Insights: Identify newly introduced, missing, or discontinued brands, as well as leading brands within chosen categories.
    • Product Insights: Identify bestsellers and evaluate their impact on your portfolio, analyzing pricing and promotions.
    • Personalized Recommendations: Receive suggestions tailored to your behavior and user profile to refine decision-making.
    • User-Configured Alerts: Stay informed with alerts designed to highlight significant changes or opportunities.

    The platform addresses data overload by providing an intuitive, insight-driven view of your assortment. Category managers gain a comprehensive, bird’s-eye perspective of key changes within specified timeframes, allowing them to focus on what matters most.

    Preparing for the Future of Retail Health

    To avoid supply chain bottlenecks, inventory shortages, and out-of-stock scenarios, retailers are strategically using data insights to anticipate fluctuations in demand and proactively plan how to manage disruptions that could affect their assortments.

    For variety that satisfies consumers’ diverse product needs, retailers are using data insights to determine whether to collaborate with nimble suppliers to promptly fill any gaps.

    To further strengthen their assortments’ attractiveness, retailers are using AI-powered pricing analytics to offer the right product at the right price. These analytics help retailers know exactly how they compare to rivals’ pricing moves with relevant data so they can keep up with market fluctuations and stay competitive by earning consumer engagement, sales, and trust.

    To Conclude

    Like nourishing habits that improve consumers’ health, data insights improve retailers’ e-commerce health. Advanced assortment and pricing analytics, powered by artificial intelligence, help retailers make better decisions faster to boost their agility, outmaneuver rivals, and fuel online growth.

    In a retail landscape where consumer preferences for health and wellness continue to evolve rapidly, the retailers who thrive will be those who leverage data and AI to understand, anticipate, and meet these changing demands with the right products at the right time. Reach out to us to know more.

  • Using Siamese Networks to Power Accurate Product Matching in eCommerce

    Using Siamese Networks to Power Accurate Product Matching in eCommerce

    Retailers often compete on price to gain market share in high performance product categories. Brands too must ensure that their in-demand assortment is competitively priced across retailers. Commerce and digital shelf analytics solutions offer competitive pricing insights at both granular and SKU levels. Central to this intelligence gathering is a vital process: product matching.

    Product matching or product mapping involves associating identical or similar products across diverse online platforms or marketplaces. The matching process leverages the capabilities of Artificial Intelligence (AI) to automatically create connections between various representations of identical or similar products. AI models create groups or clusters of products that are exactly the same or “similar” (based on some objectively defined similarity criteria) to solve different use cases for retailers and consumer brands.

    Accurate product matching offers several key benefits for brands and retailers:

    • Competitive Pricing: By identifying identical products across platforms, businesses can compare prices and adjust their strategies to remain competitive.
    • Market Intelligence: Product matching enables brands to track their products’ performance across various retailers, providing valuable insights into market trends and consumer preferences.
    • Assortment Planning: Retailers can analyze their product range against competitors, identifying gaps or opportunities in their offerings.

    Why Product Matching is Incredibly Hard

    But product matching stands out as one of the most demanding technical processes for commerce intelligence tools. Here’s why:

    Data Complexity

    Product information comes in various (multimodal) formats – text, images, and sometimes video. Each format presents its own set of challenges, from inconsistent naming conventions to varying image quality.

    Data Variance

    The considerable fluctuations in both data quality and quantity across diverse product categories, geographical regions, and websites introduce an additional layer of complexity to the product matching process.

    Industry Specific Nuances

    Industry specific nuances introduce unique challenges to product matching. Exact matching may make sense in certain verticals, such as matching part numbers in industrial equipment or identifying substitute products in pharmaceuticals. But for other industries, exactly matched products may not offer accurate comparisons.

    • In the Fashion and Apparel industry, style-to-style matching, accommodating variants and distinguishing between core sizes and non-core sizes and age groups become essential for accurate results.
    • In Home Improvement, the presence of unbranded products, private labels, and the preference for matching sets rather than individual items complicates the process.
    • On the other hand, for grocery, product matching becomes intricate due to the distinction between item pricing and unit pricing. Managing the diverse landscape of different pack sizes, quantities, and packaging adds further layers of complexity.

    Diverse Downstream Use Cases

    The diverse downstream business applications give rise to various flavors of product matching tailored to meet specific needs and objectives.

    In essence, while product matching is a critical component in eCommerce, its intricacies demand sophisticated solutions that address the above challenges.

    To solve these challenges, at DataWeave, we’ve developed an advanced product matching system using Siamese Networks, a type of machine learning model particularly suited for comparison tasks.

    Siamese Networks for Product Matching

    Our methodology involves the use of ensemble deep learning architectures. In such cases, multiple AI models are trained and used simultaneously to ensure highly accurate matches. These models tackle NLP (natural language processing) and Computer Vision challenges specific to eCommerce. This technology helps us efficiently narrow down millions of product candidates to just 5-15 highly relevant matches.

    The Tech Powering Siamese Networks

    The key to our approach is creating what we call “embeddings” – think of these as unique digital fingerprints for each product. These embeddings are designed to capture the essence of a product in a way that makes similar products easy to identify, even when they look slightly different or have different names.

    Our system learns to create these embeddings by looking at millions of product pairs. It learns to make the embeddings for similar products very close to each other while keeping the embeddings for different products far apart. This process, known as metric learning, allows our system to recognize product similarities without needing to put every product into a rigid category.

    This approach is particularly powerful for eCommerce, where we often need to match products across different websites that might use different names or images for the same item. By focusing on the key features that make each product unique, our system can accurately match products even in challenging situations.

    How Siamese Networks Work?

    Imagine having a pair of identical twins who are experts at spotting similarities and differences. That’s essentially what a Siamese network is – a pair of identical AI systems working together to compare things.

    How it works:

    • Twin AI systems: Two identical AI systems look at two different products.
    • Creating ‘fingerprints’ or ‘embedding’: Each system creates a unique ‘fingerprint’ of the product it’s looking at.
    • Comparison: These ‘fingerprints’ are then compared to see how similar the products are.

    Architecture

    The architecture of a Siamese network typically consists of three main components: the shared network, the similarity metric, and the contrastive loss function.

    • Shared Network: This is the ‘brain’ that creates the product ‘fingerprints’ or ‘embeddings.’ It is responsible for extracting meaningful feature representations from the input samples. This network is composed of layers of neural units that work together. Weight sharing between the twin networks ensures that the model learns to extract comparable features for similar inputs, providing a basis for comparison.
    • Similarity Metric: After the shared network processes the inputs, a similarity metric is employed. This decides how alike two ‘fingerprints’ or ‘embeddings’ are. The selection of a similarity metric depends on the specific task and characteristics of the input data. Frequently used similarity metrics include the Euclidean distance, cosine similarity, or correlation coefficient, each chosen based on its suitability for the given context and desired outcomes.
    • Loss Function: For training the Siamese network, a specialized loss function is used. This helps the system improve its comparison skills over time. It guides and trains the network to generate akin embeddings for similar inputs and disparate embeddings for dissimilar inputs.

      This is achieved by imposing penalties on the model when the distance or dissimilarity between similar pairs surpasses a designated threshold, or when the distance between dissimilar pairs falls below another predefined threshold. This training strategy ensures that the network becomes adept at discerning and encoding the desired level of similarity or dissimilarity in its learned embeddings.

    How DataWeave Uses Siamese Networks for Product Matching

    At DataWeave, we use Siamese Networks to match products across different retailer websites. Here’s how it works:

    Pre-processing (Image Preparation)

    • We collect product images from various websites.
    • We clean these images up to make them easier for our AI to understand.
    • We use techniques like cropping, flipping, and adjusting colors to help our AI recognize products even if the images are slightly different.

    Training The AI

    • We show our AI system millions of product images, teaching it to recognize similarities and differences.
    • We use a special learning method called “Triplet Loss” to help our AI understand which products are the same and which are different.
    • We’ve tested different AI structures to find the one that works best for product matching, including ResNet, EfficientNet, NFNet, and ViT. 

    Image Retrieval 

    • Once trained, our AI creates a unique “fingerprint” for each product image.
    • We store these fingerprints in a smart database.
    • When we need to find a match for a product, we:
      • Create a fingerprint for the new product.
      • Quickly search our database for the most similar fingerprints.
      • Return the top matching products.

    Matches are then assigned a high or a low similarity score and segregated into “Exact Matches” or “Similar Matches.” For example, check out the image of this white shoe on the left. It has a low similarity score with the pink shoe (below) and so these SKUs are categorized as a “Similar Match.” Meanwhile, the shoe on the right is categorized as an “Exact Match.”

    Similarly, in the following image of the dress for a young girl, the matched SKU has a high similarity score and so this pair is categorized as an “Exact Match.”

    Siamese Networks play a pivotal role in DataWeave’s Product Matching Engine. Amid the millions of images and product descriptions online, our Siamese Networks act as an equalizing force, efficiently narrowing down millions of candidates to a curated selection of 10-15 potential matches. 

    In addition, these networks also find application in several other contexts at DataWeave. They are used to train our system to understand text-only data from product titles and joint multimodal content from product descriptions.

    Leverage Our AI-Driven Product Matching To Get Insightful Data

    In summary, accurate and efficient product matching is no longer a luxury – it’s a necessity. DataWeave’s advanced product matching solution provides brands and retailers with the tools they need to navigate this complex landscape, turning the challenge of product matching into a competitive advantage.

    By leveraging cutting-edge technology and simplifying it for practical use, we empower businesses to make informed decisions, optimize their operations, and stay ahead in the ever-evolving eCommerce market. To learn more, reach out to us today!

  • How Monitoring and Analyzing  End-User Prices can Help Retailers and Brands Gain a Competitive Edge

    How Monitoring and Analyzing  End-User Prices can Help Retailers and Brands Gain a Competitive Edge

    Retailers and brands are constantly engaged in a fierce battle over prices and discounts. Whether it’s major events like Amazon Prime Day, brand-led sales, or everyday price wars, they depend on pricing intelligence and digital shelf analytics to fine-tune their strategies. With a variety of offers such as sales, promotions, and bundles, determining the actual cost to the customer becomes a complex task. The price set by the brand, the retailer’s offer, and the final amount paid by the customer often vary significantly.

    In their analysis, retailers and brands frequently focus on the listed price or the final sale price, overlooking a critical factor: the “end-user price.” This includes all discounts, taxes, and shipping costs, providing a more accurate picture of what customers are truly willing to pay at checkout.

    Grasping this end-user price is vital for both retailers and brands. For retailers, it helps them stay competitive and refine their promotional strategies. For brands, it offers insights into competitive positioning, net revenue management, and shaping customer price perception.

    However, emphasizing the end-user price is challenging, as it involves comprehending all the intricate elements of pricing.

    How end-user pricing is calculated

    The list price, also known as the manufacturer’s recommended retail price (MSRP), is the initial price set by the brand. This may not always be displayed on marketplaces, especially in categories like grocery. The selling price, on the other hand, is the amount at which a retailer offers the product, often reduced from the list price. The end-user price is the actual amount the customer pays at checkout, which includes taxes, promotions, and other factors that affect the final cost.

    The process involves 3 key stages:

    Step 1: Identifying and categorizing promotional offers

    The first critical step in calculating end-user pricing is to identify and categorize the various promotional offers available for a given product that can reduce the final amount paid by the consumer. These promotions span a wide range of types:

    • Bank Offers: Involving discounts or cash back incentives when paying with specific bank credit or debit cards. For instance, a customer may receive 10% cashback on their purchase by using a specific bank’s card.
    • Bundled Deals: Combining multiple products or services at a discounted bundle price. A common example is a smartphone bundle including the phone itself, a protective case, and earphones at a reduced total cost.
    • Promo Codes/Coupons: Customers can enter promo codes or coupons during checkout to unlock special discounted prices or percentage-off offers, like 20% off a hotel booking, or even a special brand discount personalized for their needs (think loyalty offers and in-app promotions).
    • Shipping Offers: These include free shipping or reduced shipping fees for certain products or orders, such as free delivery on orders above a set amount.
    • TPRs (Temporary Price Reductions): TPRs play a significant role in the strategies of most retailers. Brands and retailers use them to encourage shoppers to purchase more of a product or to try a new product they wouldn’t usually buy. A TPR involves reducing the price of a product by more than 5% from its regular shelf price.

    By accurately identifying and classifying each type of promotion available, brands can then calculate the potential end-user pricing points.

    Step 2: Accounting for location and fulfilment nuances (delivery, in-store pickup) that impact final pricing

    Product pricing and promotional offers can vary based on the consumer’s location or ZIP code. Additionally, customers may opt for different fulfilment modes like delivery, shipping, or in-store pickup, which can further impact the final cost. Accurately calculating the end-user price necessitates considering these location-based pricing nuances as well as the chosen fulfilment method.

    In the example below, the selling price is $4.32 for one retailer (on the left in the image) after a discount for online purchase. In another case with Meijer, the item total shows $17.91, but the consumer ends up paying $15.74 after taxes and fees are applied (on the right in the image).

    Step 3: Applying each eligible promotion or offer to the selling price to determine potential end-user price points

    With the various promotional offers and discounts categorized in the previous steps, retailers and brands can now apply each eligible promotion to the product’s selling price. This involves deducting percentages for bank cashback, implementing bundled pricing, applying coupon code discounts, and incorporating shipping promotions.

    For retailers, this step allows them to calculate their true effective selling price to customers after all discounts and promotions. They can then compare this end-user price against competitors to ensure they remain competitively priced.

    For brands, by systematically layering every applicable offer onto the baseline selling price, they can accurately calculate the multiple potential end-user price points a customer may pay at checkout for their products across different retailers and regions.

    Why the end-user price matters

    Optimizing pricing strategies using the end-user price can benefit retailers and brands in several ways:

    • Price Competitiveness: By monitoring end-user pricing, retailers can adjust for discounts and promotional offers to attract customers, while brands can refine their pricing models to stay ahead in the market.
    • Customer Acquisition and Loyalty: Offers, promotions, and discounts directly impact the final price paid by customers, playing a crucial role in attracting new customers and retaining existing ones. For example, Walmart’s competitive pricing in groceries boosts customer loyalty and repeat purchases.
    • Consumer Perception: End-user pricing significantly shapes how consumers perceive both retailers and brands. Competitive pricing and promotional transparency enhance reputation and conversion rates. Amazon, for instance, is known for its competitive pricing and fast deliveries, which strengthen its consumer perception and satisfaction.
    • Sales Volumes: The final checkout price influences affordability and perceived value, directly affecting sales volumes. Both retailers and brands benefit from understanding this, as it guides consumer purchasing decisions and drives revenue streams.
    • Brand Perception: Consistent and transparent pricing enhances the perception of both the retailer and the brand. This not only strengthens the value proposition but also builds consumer trust and fosters long-term loyalty.

    While the listed and selling prices are readily available, calculating the true end-user price is quite complex. It involves meticulous tracking and application of various types of promotions, offers, location-based pricing nuances, and fulfillment costs – an uphill task without robust technological solutions.

    Track and Analyze end-user prices with DataWeave

    DataWeave’s end-user price tracking capability empowers retailers and brands with the insights and tools necessary to comprehend the complexities of pricing dynamics. For retailers, it offers the ability to monitor end-user pricing across various products and categories compared to competitors, ensuring competitiveness after all discounts and enabling optimization of promotional strategies. Brands benefit from informed pricing decisions, optimized strategies across retail channels, and a strengthened position within their industries.

    Our intuitive dashboard presents classified promotions and corresponding end-user prices across retailers, providing both retailers and brands with a transparent, comprehensive view of the end-user pricing landscape.

    Within the detailed product view of DataWeave’s dashboard, the Price and Promotions panel showcases diverse promotions available across different retailers for each product, along with the potential end-user price post-promotions.

    Harness the power of DataWeave’s sophisticated Pricing Intelligence and Digital Shelf Analytics to gain an accurate, real-time understanding of your end-user pricing dynamics. Make data-driven pricing decisions that resonate with customers and propel your brand toward sustained success.

    Find out how DataWeave can empower your eCommerce pricing strategy – get in touch with us today or write to us at contact@dataweave.com!

  • Augmenting AI-powered Product Matching with Human Expertise to Achieve Unparalleled Accuracy

    Augmenting AI-powered Product Matching with Human Expertise to Achieve Unparalleled Accuracy

    In today’s expansive omnichannel commerce landscape, pricing intelligence has become indispensable for retailers seeking to stay competitive and refine their pricing strategies. The sheer magnitude of eCommerce, spanning thousands of websites, billions of SKUs, and various form factors, adds layers of complexity. Consequently, ensuring the accuracy and reliability of competitive insights presents a formidable challenge for retailers aiming to leverage pricing data effectively.

    At the core of any robust pricing intelligence system lies product matching. This process enables retailers to recognize identical or similar products across competitors. Once these matches are identified, tracking prices is a relatively more straightforward task, facilitating ongoing analysis and informed decision-making.

    Accurate matching is crucial for meaningful price comparisons and tailoring product assortments. The challenge is matching products is often complicated, especially for non-local brands, niche categories, or items lacking consistent global identifiers. It becomes even trickier when trying to match very similar but not identical products. A comprehensive approach that compares and analyzes multiple attributes like product titles, descriptions, images and more is essential.

    Artificial intelligence algorithms are commonly used to automate product matching, leveraging machine learning techniques to analyze patterns in images and text data. While AI can adapt and improve over time, the question remains: Can it fully address the complexities of product matching on its own?

    The reality is that many retailers still struggle with incomplete, inaccurate, or outdated product data, despite these AI-powered product matching solutions. This can lead to suboptimal pricing decisions, missed opportunities, and reduced competitiveness.

    Challenges in an ‘AI-only’ Approach to Product Matching

    While AI plays a vital role in automated product matching solutions, there are complexities that AI alone cannot fully address:

    Subjectivity in Matching Criteria

    Some product categories have subjective or hard-to-quantify criteria for determining similarity. AI learns from historical data, so it may struggle with nuanced aspects like:

    Aesthetics, style, and design: In the Fashion and Jewellery vertical, for example, products are matched according to attributes like style, aesthetics, design – all of which have some subjectivity involved.

    Quantity/packaging variations: In the grocery sector, variations in product packaging and quantities can introduce complexities that require subjective decision-making. For example, apples may be sold in different packaging like a 0.5 kg bag or a pack of 4 individual apples. Determining if these different packaging options should be considered equivalent often involves making a qualitative judgment call, rather than a clear-cut objective decision.

    Matching product sets: For categories like home furnishings, the focus is often on matching coordinated sets rather than individual items. For example, in the bedroom category, matching may involve grouping together an entire set of complementary furniture like a bed frame, dresser, and wardrobe based on their cohesive design and style. This goes beyond simply making one-to-one product associations, requiring more nuanced judgments about aesthetic coordination.

    Contextual Factor

    Products can have regional preferences, cultural differences, or evolving trends that impact how they are matched. AI may miss important context like Local/regional product names or distinct brand names across countries.

    For instance, in the image we see Sprite (in the US) is branded Xubei in China. Continuous human curation is needed to help AI adapt to this context.

    High Accuracy & Coverage Expectations

    Retailers rely on AI powered and automated pricing adjustments based on product matching for insight. To ensure that pricing recommendations and updates are accurate, accurate product matching is crucial. For this, simply identifying similar top results is not enough – the process must comprehensively capture all relevant matches. While AI excels at finding the top groupings with around 80% accuracy, even small matching errors can have significant consequences.

    As AI matching improves, customer expectations may rise even higher. If AI achieves 90% accuracy, for instance, SLAs may demand over 95%. Reaching such a high level of accuracy is very challenging for AI alone, especially when faced with incomplete data, contextual nuances, evolving trends, and subjective matching criteria across products and categories.

    The solution is to combine the power of AI with human expertise. This is the key to achieving true data veracity – the accuracy, freshness, and comprehensive coverage required for precise and reliable product matching.

    Human-in-the-Loop Approach for Elevated Product Matching

    Human intelligence and quality testing can elevate the AI powered product matching process by addressing key challenges:

    • Matching Validation: AI algorithms may identify product matches with 80-90% accuracy initially. Having humans validate these AI-suggested matches allows for correcting errors and pushing the accuracy close to 100%. As humans flag issues, provide context, and re-label incorrect predictions, it allows the AI model to learn and enhance its reliability for complex, high-stakes decisions.
    • Applying Contextual Judgment: For subjective matching criteria like aesthetics, design, and categorizing product sets, human discernment is needed. Humans can make nuanced judgments beyond just quantitative rules, ensuring meaningful apples-to-apples product comparisons. Their contextual understanding augments AI’s capabilities.
    • Continuous Learning Via Feedback Loop: Product experts possess rich category knowledge across markets. Integrating this human insight through an iterative feedback loop helps AI models quickly learn and adapt to changing trends, preferences, and context. As humans explain their match assessments, the AI continuously enhances its precision over time.

    By combining AI’s automation and scale with human validation, judgment, and knowledge curation, pricing intelligence solutions can achieve the accuracy and coverage demanded for actionable competitive pricing insights.

    DataWeave’s Data Veracity Framework: A Scalable Workflow Combining AI and Human Expertise

    Given the vast number of products, retailers, and brands that exist today, any product matching solution must be highly scalable. At DataWeave, we bring you such a scalable workflow to address these complexities by integrating human expertise with AI-driven automation. The image below outlines our approach for combining AI with human intelligence in a seamless, scalable workflow for accurate product matching:

    Retailers and brands can benefit in several ways with this workflow, as listed below.

    Several Rounds of Data Verification Due to Hierarchical Validation Teams

    The workflow employs a hierarchical validation team of Leads and Executives to efficiently integrate human expertise without creating bottlenecks. Verification Leads play a pivotal role in managing the distribution of product matches identified by DataWeave’s AI model to the Verification Executives.

    The Executives then meticulously validate these AI-suggested matches, adding any missing product associations and removing inaccurate matches. After validation, the matched product groups are sent back to the Leads, who perform random sampling checks to ensure quality.

    Throughout this entire workflow, feedback and suggestions are continuously gathered from both the Executives and Leads. This curated input is then incorporated back into DataWeave’s AI model, allowing it to learn and improve its matching accuracy on an ongoing basis.

    This hierarchical structure ensures that human validation seamlessly scales alongside the AI’s matching capabilities. Leveraging the respective strengths of AI automation and human expertise in an iterative feedback loop prevents operational bottlenecks while steadily elevating overall accuracy.

    Confidence-based Distribution of Matched Articles for Validation

    The AI model assigns confidence scores, differentiating high-confidence (>95%) and low-confidence matches. For high-confidence groups, executives simply remove incorrect matches – a quicker process. Low-confidence matches require more human effort in adding/removing matches.

    As the AI model improves over time with feedback, the share of high-confidence matches increases, making validation more efficient and swift.

    Automated, Standardized Process with Iterative Feedback Loop

    The entire workflow is standardized and automated, with verification metrics seamlessly tracked. At each step, feedback captured from both leads and executives flows back into the AI, enhancing its matching accuracy and coverage iteratively.

    DataWeave’s closed-loop system of AI automation with hierarchical human validation allows product matching to achieve comprehensive accuracy at a vast scale.

    Unleash the Power Accurate and Comprehensive Product Matching

    In summary, combining AI and human expertise in product matching is crucial for retailers navigating the complexities of omnichannel retail. While AI algorithms excel in automation, they often struggle with subjective criteria and contextual nuances. DataWeave’s approach integrates AI-driven automation with human validation, delivering the industry’s most accurate product matching capabilities, enabling actionable competitive pricing insights.

    To learn more, reach out to us today!

  • How AI-Powered Visual Highlighting Helps Brands Achieve Product Consistency Across eCommerce

    How AI-Powered Visual Highlighting Helps Brands Achieve Product Consistency Across eCommerce

    As eCommerce increasingly becomes a prolific channel of sales for consumer brands, they find that maintaining a consistent and trustworthy brand image is a constant struggle. In an ecosystem filled with dozens of marketplaces and hundreds of third-party merchants, ensuring that customers see what aligns with a brand’s intended image is quite tricky. With many fakes and counterfeit products doing the rounds, brands may further struggle to get the right representation.

    One way brands can track and identify inconsistencies in their brand representation across marketplaces is to use Digital Shelf Analytics solutions like DataWeave’s – specifically the Content Audit module.

    This solution uses advanced AI models to identify image similarities and dissimilarities compared with the original brand image. Brands could then use their PIM platform or work with the retailer to replace inaccurate images.

    But here’s the catch – AI can’t always accurately predict all the differences. Relying solely on scores given by these models poses a challenge in tracking the subtle differences between images. Often, image pairs with seemingly high match scores fail to catch important distinctions. Fake or counterfeit products and variations that slip past the AI’s scrutiny can lead to significant inaccuracies. Ultimately, it puts the reliability of the insights that brands depend on for crucial decisions at risk, impacting both top and bottom lines.

    Dealing with this challenge means finding a balance between the number-based assessments of AI models and the human touch needed for accurate decision-making. However, giving auditors the ability to pinpoint variations precisely goes beyond simply sharing numerical values of the match scores with them. Visualizing model-generated scores is important as it provides human auditors with a tangible and intuitive understanding of the differences between two images. While numerical scores are comparable in the relative sense, they lack specificity. Visual interpretation empowers auditors to identify precisely where variations occur, aiding in efficient decision-making.

    How AI-Powered Image Scoring Works

    At DataWeave, our approach involves employing sophisticated computer vision models to conduct extensive image comparisons. Convolutional Neural Network (CNN) models such as Resnet-50 or YOLO, in conjunction with feature extraction models, analyze images quantitatively. This AI-powered image scoring process yields scores that indicate the level of similarity between images.

    However, interpreting these scores and understanding the specific areas of difference can be challenging for human auditors. While computer vision models excel at processing vast amounts of data quickly, translating their output into actionable insights can be a stumbling block. A numerical score may not immediately convey the nature or extent of the differences between images

    In the assessment of these images, all fall within the 70 to 80 range of scores (out of a maximum of 100). However, discerning the nature of differences—whether they are apparent or subtle—poses a challenge for the AI models and human auditors. For example, there are differences in the placement or type of images in the packaging, as well as packing text that are often in an extremely small font size. It is, of course, possible for human auditors to identify the differences in these images, but it’s a slow, error-prone, and tiring process, especially when auditors often have to check hundreds of image pairs each day.

    So how do we ensure that we identify differences in images accurately? The answer lies in the process of visual highlighting.

    How Visual Highlighting Works

    Visual highlighting is a method that enhances our ability to comprehend differences in images by combining sophisticated algorithms with human understanding. Instead of relying solely on numerical scores, this approach introduces a visual layer, resembling a heatmap, guiding human auditors to specific areas where discrepancies are present.

    Consider the scenario depicted in the images above: a computer vision model assigns a score of 70-85 for these images. While this score suggests relatively high similarity, it fails to uncover major differences between the images. Visual highlighting comes into play to overcome this limitation, precisely indicating regions where even subtle differences are seen.

    Visual highlighting entails overlaying compared images and emphasizing areas of difference, achieved through techniques like color coding, outlining, or shading specific regions. The significance of the difference in a particular area determines the intensity of the visual highlight.

    For instance, if there’s a change in the product’s color or a discrepancy in the packaging, these variations will be visually emphasized. This not only streamlines the auditing process but also enables human evaluators to make well-informed decisions quickly.

    Benefits of Visual Highlighting

    • Intuitive Understanding: Visual highlighting offers an intuitive method for interpreting and acting upon the outcomes of computer vision models. Instead of delving into numerical scores, auditors can concentrate on the highlighted areas, enhancing the efficiency and accuracy of the decision-making process.
    • Accelerated Auditing: By bringing attention to specific regions of concern, visual highlighting speeds up the auditing process. Human evaluators can swiftly identify and address discrepancies without the need for exhaustive image analysis.
    • Seamless Communication: Visual highlighting promotes clearer communication between automated systems and human auditors. Serving as a visual guide, it enhances collaboration, ensuring that the subtleties captured by computer vision models are effectively conveyed.

    The Way Forward

    As technology continues to evolve, the integration of visual highlighting methodologies is likely to become more sophisticated. Artificial intelligence and machine learning algorithms may play an even more prominent role in not only detecting differences but also in refining the visual highlighting process.

    The collaboration between human auditors and AI ensures a comprehensive approach to maintaining brand integrity in the ever-expanding digital marketplace. By visually highlighting differences in images, brands can safeguard their visual identity, foster consumer trust, and deliver a consistent and reliable online shopping experience. In the intricate dance between technology and human intuition, visual highlighting emerges as a powerful tool, paving the way for brands to uphold their image with precision and efficiency.

    To learn more, reach out to us today!


    (This article was co-authored by Apurva Naik)

  • How DataWeave Enhances Transparency in Competitive Pricing Intelligence for Retailers

    How DataWeave Enhances Transparency in Competitive Pricing Intelligence for Retailers

    Retailers heavily depend on pricing intelligence solutions to consistently achieve and uphold their desired competitive pricing positions in the market. The effectiveness of these solutions, however, hinges on the quality of the underlying data, along with the coverage of product matches across websites.

    As a retailer, gaining complete confidence in your pricing intelligence system requires a focus on the trinity of data quality:

    • Accuracy: Accurate product matching ensures that the right set of competitor product(s) are correctly grouped together along with yours. It ensures that decisions taken by pricing managers to drive competitive pricing and the desired price image are based on reliable apples-to-apples product comparisons.
    • Freshness: Timely data is paramount in navigating the dynamic market landscape. Up-to-date SKU data from competitors enables retailers to promptly adjust pricing strategies in response to market shifts, competitor promotions, or changes in customer demand.
    • Product matching coverage: Comprehensive product matching coverage ensures that products are thoroughly matched with similar or identical competitor products. This involves accurately matching variations in size, weight, color, and other attributes. A higher coverage ensures that retailers seize all available opportunities for price improvement at any given time, directly impacting revenues and margins.

    However, the reality is that untimely data and incomplete product matches have been persistent challenges for pricing teams, compromising their pricing actions. Inaccurate or incomplete data can lead to suboptimal decisions, missed opportunities, and reduced competitiveness in the market.

    What’s worse than poor-quality data? Poor-quality data masquerading as accurate data.

    In many instances, retailers face a significant challenge in obtaining comprehensive visibility into crucial data quality parameters. If they suspect the data quality of their provider is not up to the mark, they are often compelled to manually request reports from their provider to investigate further. This lack of transparency not only hampers their pricing operations but also impedes the troubleshooting process and decision-making, slowing down crucial aspects of their business.

    We’ve heard about this problem from dozens of our retail customers for a while. Now, we’ve solved it.

    DataWeave’s Data Statistics and SKU Management Capability Enhances Data Transparency

    DataWeave’s Data Statistics Dashboard, offered as part of our Pricing Intelligence solution, enables pricing teams to gain unparalleled visibility into their product matches, SKU data freshness, and accuracy.

    It enables retailers to autonomously assess and manage SKU data quality and product matches independently—a crucial aspect of ensuring the best outcomes in the dynamic landscape of eCommerce.

    Beyond providing transparency and visibility into data quality and product matches, the dashboard facilitates proactive data quality management. Users can flag incorrect matches and address various data quality issues, ensuring a proactive approach to maintaining the highest standards.

    Retailers can benefit in several ways with this dashboard, as listed below.

    View Product Match Rates Across Websites

    The dashboard helps retailers track match rates to gauge their health. High product match rates signify that pricing teams can move forward in their pricing actions with confidence. Low match rates would be a cause for further investigation, to better understand the underlying challenges, perhaps within a specific category or competitor website.

    Our dashboard presents both summary statistics on matches and data crawls as well as detailed snapshots and trend charts, providing users with a holistic and detailed perspective of their product matches.

    Additionally, the dashboard provides category-wise snapshots of reference products and their matching counterparts across various retailers, allowing users to focus on areas with lower match rates, investigate underlying reasons, and develop strategies for speedy resolution.

    Track Data Freshness Easily

    The dashboard enables pricing teams to monitor the timeliness of pricing data and assess its recency. In the dynamic realm of eCommerce, having up-to-date data is essential for making impactful pricing decisions. The dashboard’s presentation of freshness rates ensures that pricing teams are armed with the latest product details and pricing information across competitors.

    Within the dashboard, users can readily observe the count of products updated with the most recent pricing data. This feature provides insights into any temporary data capture failures that may have led to a decrease in data freshness. Armed with this information, users can adapt their pricing decisions accordingly, taking into consideration these temporary gaps in fresh data. This proactive approach ensures that pricing strategies remain agile and responsive to fluctuations in data quality.

    Proactively Manage Product Matches

    The dashboard provides users with proactive control over managing product matches within their current bundles via the ‘Data Management’ panel. This functionality empowers users to verify, add, flag, or delete product matches, offering a hands-on approach to refining the matching process. Despite the deployment of robust matching algorithms that achieve industry-leading match rates, occasional instances may arise where specific matches are overlooked or misclassified. In such cases, users play a pivotal role in fine-tuning the matching process to ensure accuracy.

    The interface’s flexibility extends to accommodating product variants and enables users to manage product matches based on store location. Additionally, the platform facilitates bulk match uploads, streamlining the process for users to efficiently handle large volumes of matching data. This versatility ensures that users have the tools they need to navigate and customize the matching process according to the nuances of their specific product landscape.

    Gain Unparalleled Visibility into your Data Quality

    With DataWeave’s Pricing Intelligence, users gain the capability to delve deep into their product data, scrutinize match rates, assess data freshness, and independently manage their product matches. This approach is instrumental in fostering informed and effective decisions, optimizing inventory management, and securing a competitive edge in the dynamic world of online retail.

    To learn more, reach out to us today!

  • Capturing and Analyzing Retail Mobile App Data for Digital Shelf Analytics: Are Brands Missing Out?

    Capturing and Analyzing Retail Mobile App Data for Digital Shelf Analytics: Are Brands Missing Out?

    Consumer brands around the world increasingly recognize the vital role of tracking and optimizing their digital shelf KPIs, such as Content Quality, Share of Search, Availability, etc. These metrics play a crucial role in boosting eCommerce sales and securing a larger online market share. With the escalating requirements of brands, the sophistication of top Digital Shelf Analytics providers is also on the rise. Consequently, the adoption of digital shelf solutions has become an essential prerequisite for today’s leading brands.

    As brands and vendors continue to delve further and deeper into the world of Digital Shelf Analytics, a significant and often overlooked aspect is the analysis of digital shelf data on mobile apps. The ability of solution providers to effectively track and analyze this mobile-specific data is crucial.

    Why is this emphasis on mobile apps important?

    Today, the battle for consumer attention unfolds not only on desktop web platforms but also within the palm of our hands – on mobile devices. As highlighted in a recent Insider Intelligence report, customers will buy more on mobile, exceeding 4 in 10 retail eCommerce dollars for the first time.

    Moreover, thanks to the growth of delivery intermediaries like Instacart, DoorDash, Uber Eats, etc., shopping on mobile apps has received a tremendous organic boost. According to an eMarketer report, US grocery delivery intermediary sales are expected to reach $68.2 billion in 2025, from only $8.8 billion in 2019.

    In essence, mobile is increasingly gaining share as the form factor of choice for consumers, especially in CPG. In fact, one of our customers, a leading multinational CPG company, revealed to us that it sees up to 70% of its online sales come through mobile apps. That’s a staggering number!

    The surge in app usage reflects a fundamental change in consumer behavior, emphasizing the need for brands to adapt their digital shelf strategies accordingly.

    Why Brands Need To Look at Apps and Desktop Data Differently

    Conventionally, brands that leverage digital shelf analytics rely on data harnessed from desktop sites of online marketplaces. This is because capturing data reliably and accurately from mobile apps is inherently complex. Data aggregation systems designed to scrape data from web applications cannot easily be repurposed to capture data on mobile apps. It requires dedicated effort and exceptional tech prowess to pull off in a meaningful and consistent way.

    In reality, it is extremely important for brands to track and optimize their mobile digital shelf. Several digital shelf metrics vary significantly between desktop sites and mobile apps. These differences are natural outcomes of differences in user behavior between the two form factors.

    One of these metrics that has a huge impact on a brand’s performance on retail mobile apps is their search discoverability. Ecommerce teams are well aware of the adverse impact of the loss of even a few ranks on search results.

    Anyone can easily test this. Searching something as simple as “running shoes” on the Amazon website and doing the same on its mobile app shows at least a few differences in product listings among the top 20-25 ranks. There are other variances too, such as the number of sponsored listings at the top, as well as the products being sponsored. These variations often result in significant differences in a brand’s Share of Search between desktop and mobile.

    Share of Search is the share of a brand’s products among the top 20 ranked products in a category or subcategory, providing insight into a brand’s visibility on online marketplaces.

    Picture a scenario in which a brand heavily depends on desktop digital shelf data, confidently assuming it holds a robust Share of Search based on reports from its Digital Shelf Analytics partner. However, unbeknownst to the team, the Share of Search on mobile is notably lower, causing a detrimental effect on sales.

    To fully understand the scale of these differences, we decided to run a small experiment using our proprietary data analysis and aggregation platform. We restricted our analysis to just Amazon.com and Amazon’s mobile app. However, we did cover over 13,000 SKUs across several shopping categories to ensure the sample size is strong.

    Below, we provide details of our key findings.

    Share of Search on The Digital Shelf – App Versus Desktop

    Our analysis focused on three popular consumer categories – Electronics, CPG, and Health & Beauty.

    In the electronics category, brands like Apple, Motorola, and Samsung, known for their mobile phones, earbuds, headphones, and more, have a higher Share of Search on the Amazon mobile app compared to the desktop.

    Meanwhile, Laptop brands like Dell, Acer, and Lenovo, as well as other leading brands like Google have a higher Share of Search on the desktop site compared to the app. This is the scenario that brands need to be careful about. When their Share of Search on mobile apps is lower, they might miss the chance to take corrective measures since they lack the necessary data from their provider.

    In the CPG category, Ramen brand Samyang, with a lot of popularity on Tiktok and Instagram, shows a higher Share of Search on Amazon’s mobile app. Speciality brands like 365 By Whole Foods, pasta and Italian food brands La Moderna, Divinia, and Bauducco too have a significantly higher Share of Search on the app.

    Cheese and dessert brands like Happy Belly, Atlanta Cheesecake Company, among others, have a lower Share of Search on the mobile app. Ramen brand Sapporo is also more easily discovered on Amazon’s desktop site. Here, we see a difference of more than 5% in the Share of Search of some brands, which is likely to have a huge impact on the brand’s mobile eCommerce sales levels and overall performance.

    Lastly, in the Health & Beauty category, Shampoos and hair care brands like Olaplex, Dove, and Tresemme exhibited a higher Share of Search on the mobile app compared to the desktop.

    On the other hand, body care brands like Neutrogena and Hawaiian Tropic, as well as Beardcare brand Viking Revolution displayed a higher Share of Search on Amazon’s desktop site.

    Based on our data, it is clear that there are several examples of brands that do better in either one of Amazon’s desktop sites or mobile apps. In many cases, the difference is stark.

    So What Must Brands Do?

    Our findings emphasize the imperative for brands to move beyond a one-size-fits-all approach to digital shelf analytics. The striking variations in Share of Search between mobile apps and desktops conclusively demonstrate that relying solely on desktop data for digital shelf optimization is inadequate.

    If brands see that they’re falling behind on the mobile digital shelf, there are a few things they can do to help boost their performance:

    • If a brand’s Share of Search is lower on the mobile app, they can divert their retail spend to mobile in order to inorganically compensate for this difference. This way, any short-term impact due to lower discoverability is mitigated. This is also likely to result in optimized budget allocation and ROAS.
    • Brands also need to ensure their content is optimized for the mobile form factor, with images that are easy to view on smaller screens, and tailored product titles that are shorter than on desktops, highlighting the most important product attributes from the consumer’s perspective. Not only will this help brands gain more clicks from mobile shoppers, but this will also gradually lead to a boost in their organic Share of Search on mobile.
    • CPG brands, specifically, need to optimize their digital shelf for delivery intermediary apps (along with marketplaces). The grocery delivery ecosystem is booming with companies like DoorDash, Delivery Hero, Uber Eats, Swiggy, etc. leading the way. Using Digital Shelf Analytics to optimize performance on delivery apps is quite an involved process with a lot of bells and whistles to consider. Read our recently published whitepaper that specifically details how brands can successfully boost their visibility and conversions on delivery apps.

    But first, brands need to identify and work with a Digital Shelf Analytics partner that is able to capture and analyze mobile app data, enabling tailored optimization approaches for all eCommerce platforms.

    DataWeave leads the way here, providing the world’s most comprehensive and sophisticated digital shelf analytics solution, rising above all other providers to provide digital shelf insights for both web applications and mobile apps. Our data aggregation platform successfully navigates the intricacies of capturing public data accurately and reliably from mobile apps, thereby delivering a comprehensive cross-device view of digital shelf KPIs to our brand customers.

    So reach out to us today to find out more about our digital shelf solutions for mobile apps!

  • AI-powered Product Matching: The Key to Competitive Pricing Intelligence in eCommerce

    AI-powered Product Matching: The Key to Competitive Pricing Intelligence in eCommerce

    With thousands of products and hundreds of online retailers to choose from, the average modern-day shopper usually compares prices across several e-commerce sites effortlessly before often settling for the lowest priced option. As a result, retailers today are forced to execute millions of price changes per day in a never-ending race to be the lowest priced – without losing out on any potential margin.

    Identifying, classifying, and matching products is the first step to comparing prices across websites. However, there is no standardization in the way products are represented across e-commerce websites, causing this process to be fairly complex.

    Here’s an example:

    What’s needed is a pricing intelligence solution that first matches products across several websites swiftly and accurately, and then enables automated tracking of competitor pricing data on an ongoing basis.

    Pricing intelligence solutions already exist. What’s wrong with using them?

    There are several challenges with the incumbent solutions in the market – the biggest one being that they don’t work in a timely manner. In essence, it’s like deferring the process of finding actionable information that helps retailers acquire a competitive advantage, and instead doing it in hindsight. Like an autopsy of sorts.

    Here are the various solution types we have in the market today:

    • Internally developed systems – Solutions developed by retailers themselves often rely on heavy manual data aggregation and have poor product matching capabilities. Since these solutions have been developed by professionals not attuned to building data crunching machines, they pose significant operational challenges in the form of maintenance, updates, etc.
    • Web scraping solutions – These solutions have no data normalization or product matching capabilities, and lack the power to deliver relevant actionable insights. What’s more, it’s a struggle to scale them up to accommodate massive volumes of data during peak times such as promotional campaigns.
    • DIY solutions – These solutions require manual research and entry of data. It goes without saying that due to the level of human intervention and effort required, they’re expensive, difficult to scale, slow, and of questionable accuracy.

    As common as it is nowadays, AI has the answer

    DataWeave’s competitive pricing intelligence solution is designed to help retailers achieve precisely the competitive advantage they need by providing them with accurate, timely, and actionable pricing insights enabled by matching products at scale. We provide retailers with access to detailed pricing information on millions of products across competitors, as frequently as they need it.

    Our technology stack broadly consists of the following.

    1. Data Aggregation

    At DataWeave, we can aggregate data from diverse web sources across complex web environments – consistently and at a very high accuracy. Having been in the industry for close to a decade, we’re sitting on a lot of data that we can use to train our product matching platform.

    Our datasets include data points from tens of millions of products and have been collected from numerous geographies and verticals in retail. The datasets contain hierarchically arranged information based on retail taxonomy. At the root level, there’s information such as category and subcategory, and at the top level, we have product details such as title, description, and other <attribute, value> relationships. Our machine learning architectures and semi-automated training data building systems, augmented by the skills of a strong QA team, help us annotate the necessary information and create labeled datasets using proprietary tools.

    2. AI for Product Matching

    Product matching at DataWeave is done via a unified platform that uses both text and image recognition capabilities to accurately identify similar SKUs across thousands of e-commerce stores and millions of products. We use an ensemble deep learning architectures tailored to NLP and Computer Vision problems specific to us and heuristics pertinent to the Retail domain. Products are also classified based on their features, and a normalization layer is designed based on various text/image-based attributes.

    Our semantics layer, while technically an integral part of the product matching process, deserves particular mention due to its powerful capabilities.

    The text data processing consists of internal, deep pre-trained word embeddings. We use state-of-the-art, customized word representation techniques such as ELMO, BERT, and Transformer to capture deeply contextualized text with improved accuracy. A self-attention/intra-attention mechanism learns the correlation between the word in question and a previous part of the description.

    Image data processing starts with object detection to identify the region of interest of a given product (for example, the upper body of a fashion model displaying a shirt). We then leverage deep learning architectures such as VggNet, Inception-V3, and ResNet, which we have trained using millions of labeled images. Next, we apply multiple pre-processing techniques such as variable background removal, face removal, skin removal, and image quality enhancing and extract image signatures via deep learning and machine learning-based algorithms to uniquely identify products across billions of indexed products.

    Finally, we efficiently distribute billions of images across multiple stores for fast access, and to facilitate searches at a massive scale (in a matter of milliseconds, without the slightest compromise on accuracy) using our image matching engine.

    3. Human Intelligence in the Loop

    In scenarios where the confidence scores of the machine-driven matches are low, we have a team of Quality Assurance (QA) specialists who verify the output.

    This team does three things:

    • Find out why the confidence score is low
    • Confirm the right product matches
    • Figure out a way to encode this knowledge into a rule and feed it back to the algorithm

    In this way, we’ve built a self-improving feedback loop which, by its very nature, performs better over time. This system has accumulated knowledge over the 8 years of our operations, which is going to be hard for anyone to replicate. Essentially, this process enables us to match products at massive scale quickly and at very high levels of accuracy (usually over 95%).

    4. Actionable Insights Via Data Visualization

    Once the matching process is completed, the prices are aggregated at any frequency, enabling retailers to optimize their prices on an ongoing basis. Pricing insights are typically consumed via our SaaS-based web-portal, which consists of dashboards, reports, and visualizations.

    Alternatively, we can integrate with internal analytics platforms through APIs or generate and deliver spreadsheet reports on a regular basis, depending on the preferences of our customers.

    To summarize

    The benefits of our solution are many. Detailed price improvement opportunity-related insights generated in a timely manner empower retailers to significantly enhance their competitive positioning across categories, product types, and brands, as well as ability to influence their price perception among consumers. These insights, when leveraged at a higher granularity over the long term, can help maximize revenue through price optimization at a large scale.

    Our solution also helps drive process-based as well as operational optimizations for retailers. Such modifications help them better align themselves to effectively adopt a data-driven approach to pricing, in turn helping them achieve much smarter retail operations across the board.

    All of this wouldn’t be possible if the product matching process, inherent to this system, was unreliable, expensive, or time-consuming.

    If you would like to learn more about DataWeave’s proprietary product matching platform and the benefits it offers to eCommerce businesses and brands, talk to us now!

  • Why Unit of Measure Normalization is Critical For Accurate and Actionable Competitive Pricing Intelligence

    Why Unit of Measure Normalization is Critical For Accurate and Actionable Competitive Pricing Intelligence

    Competitive pricing intelligence is pivotal for retailers seeking to analyze their product pricing in relation to competitors. This practice is essential for ensuring that their product range maintains a competitive edge, meeting both customer expectations and market demands consistently.

    Product matching serves as a foundational element within any competitive pricing intelligence solution. Products are frequently presented in varying formats across different websites, featuring distinct titles, images, and descriptions. Undertaking this process at a significant scale is highly intricate due to numerous factors. One such complication arises from the fact that products are often displayed with differing units of measurement on various websites.

    The Challenge of Varying Units

    In certain product categories, retailers often offer the same item in varying volumes, quantities, or weights. For instance, a clothing item might be available as a single piece or in packs of 2 or 3, while grocery brands commonly sell eggs in counts of 6, 12, or 24.

    Consider this example: a quick glance might suggest that an 850g pack of Kellogg’s Corn Flakes priced at $5 is a better deal than a 980g pack of Nestle Cornflakes priced at $5.2. However, this assumption can be deceptive. In reality, the latter offers better value for your money, a fact that only becomes evident through price comparisons after standardizing the units.

    This issue is particularly relevant due to the prevalence of “shrinkflation,” where brands adjust packaging sizes or quantities to offset inflation while keeping prices seemingly low. When quantities, pack sizes, weight, etc. reduce instead of prices increasing, it’s important that this change is considered while analyzing competitive pricing.

    Normalizing Units of Measure

    In order to effectively compare prices among different competitors, retailers must standardize the diverse units of measurement they encounter. This standardization (or normalization) is crucial because price comparisons should extend beyond individual product SKUs to accommodate variations in package sizes and quantities. It’s essential to normalize units, ranging from “each” (ea) for individual items to “dozen” (dz) for sets, and from “pounds” (lb), “kilograms” (kg), “liters” (ltr), to “gallons” (gal) for various product types.

    For example, a predetermined base unit of measure, such as 100 grams for a specific product like cornflakes, serves as the reference point. The unit-normalized price for any cornflake product would then be the price per 100 grams. In the example provided, this reveals that Kellogg’s is priced at $0.59 per 100 grams, while Nestle is priced at $0.53 per 100 grams.

    Various Categories of Unit Normalization

    1. Weight Normalization

    Retailers frequently feature products with weight measurements expressed in grams (g), kilograms (kg), pounds (lbs), or ounces (oz).

    2. Quantity or Pack Size Normalization

    Products are also often featured with varying pick sizes or quantities in each SKU.

    mounting hardware kit

    3. Volume or Capacity Normalization

    Products can also vary in volumes or capacities with units like liters (L) or fluid ounces (fl oz).

    DataWeave’s Unit Normalized Pricing Intelligence Solution

    DataWeave’s highly sophisticated product matching engine can match the same or similar products and normalize their units of measurement, leading to highly accurate and actionable competitive pricing insights. It standardizes different units of measurement, like weight, quantity, and volume, ensuring fair comparisons across similar and exact matched products.

    Retailers have the flexibility to view pricing insights either with retailer units or normalized units. This capability empowers retailers and analysts to perform accurate, in-depth analyses of pricing information at a product level.

    In some scenarios, analyzing unit normalized pricing reflects pricing trends and competitiveness more accurately than retail price alone. This is particularly true for categories like CPG, where products are sold in diverse units of measure. For instance, in the example shown here, we can view a comparison of price position trends for the category of Fruits and Vegetables based on both retail price and unit price.

    The difference is striking: the original retail price based analysis shows a stagnation in price position, whereas unit normalized pricing analysis reflects a more dynamic pricing scenario.

    With DataWeave, retailers can specify which units to compare, ensuring that comparisons are made accurately. For example, a retailer can specify that unit price comparisons apply only to 8, 12, or 16-ounce packs, as well as 1 or 3-pound packs, but not to 10 and 25-pound bags. This precision ensures that products are matched correctly, and prices are represented for appropriately normalized units, leading to more accurate pricing insights.

    To learn more about this capability, write to us at contact@dataweave.com or visit our website today!

  • From Data to Dollars: How Digital Shelf Analytics Drives Tangible Business Impact and ROI for Brands

    From Data to Dollars: How Digital Shelf Analytics Drives Tangible Business Impact and ROI for Brands

    For consumer brands, the digital marketplace presents an unparalleled landscape of opportunities for engaging with consumers and expanding their market presence. Within this dynamic environment, Digital Shelf Analytics has emerged as a crucial pillar in a brand’s eCommerce strategy. This technology provides valuable insights into a brand’s organic and paid visibility on marketplaces, content quality, pricing strategies, promotional efforts, and product availability. These insights help brands gain a comprehensive understanding of their competitive positioning and overall market performance.

    Nevertheless, many brands often grapple with the question of whether this understanding translates into tangible actions that drive real business impact and return on investment (ROI). This uncertainty stems from a lack of clarity about the direct correlation between digital shelf insights and key metrics such as enhanced sales conversions.

    Nonetheless, there is compelling evidence that when these insights are effectively harnessed and strategic actions are taken, brands can realize significant, measurable benefits.

    So, the question arises: does Digital Shelf Analytics genuinely deliver on its promises?

    At DataWeave, we’ve partnered with numerous brands to fuel their eCommerce growth through the application of digital shelf analytics. In this article, we will delve into these insights, uncovering the concrete and quantifiable results that brands can achieve through their investments in digital shelf analytics.

    Digital Shelf KPIs and Their Impact

    Digital Shelf Analytics is a robust system that analyzes specific key performance indicators (KPIs) about the digital shelf, furnishing brands with precise recommendations to not only bolster these KPIs but also to monitor the enhancements over time. The following is a brief explanation of digital shelf KPis and their expected impact areas:

    Product Availability: Ensuring Shoppers Never Hear “Out of Stock” Again

    Timely insights on the availability of products ensures brands reduce replenishment times at scale, which can significantly impact sales, creating an unbreakable link between product availability and revenue. With Digital Shelf Analytics, procurement and replenishment teams can set up notifications to promptly identify low or out-of-stock items and take swift action. This can also be done for specific ZIP codes or individual stores. In addition, availability plays a crucial role in a brand’s Share of Search and search rankings, as online marketplaces often ensure only in-stock products are shown among the top ranks.

    Share of Search: Dominating the Digital Aisles

    If a product isn’t visible, does it even exist? In fact, 70% of consumers never go beyond the first page of search results on major online marketplaces. Therefore, as a brand, the visibility of your products for relevant search keywords and their appearance on the first page can heavily determine your awareness metrics. This is where the concept of Share of Search comes into play. Think of it as securing prime shelf space in a physical store. Digital shelf insights and benchmarking with category leaders for Share of Search help ensure your products command relevant attention on the digital shelf.

    Content Quality: Crafting the Perfect Product Story

    Creating engaging product descriptions and visuals is akin to giving your products a megaphone in a crowded marketplace. By enhancing content quality, including product names, titles, descriptions, and images, brands can climb the search result rankings, leading to increased visibility and subsequently, more sales.

    Ratings and Reviews: The Power of Social Proof

    Public opinion holds immense sway. Research indicates that a single positive review can trigger a 10% surge in sales, while a multitude of favorable reviews can propel your product to a 44% higher trajectory. The correlation between ratings and sales is not surprising—each step up the rating ladder can translate to substantial revenue growth.

    While it’s reasonable to anticipate a connection between these KPIs and downstream impact metrics such as impressions, clicks, and conversions, we were driven to explore this correlation through the lens of real-world data. To do so, we meticulously monitored the digital shelf KPIs for one of our clients and analyzed the improvements in these metrics.

    It’s essential to acknowledge that not all observed impact areas can be solely attributed to enhancements in digital shelf KPIs. Still, it’s evident that a robust correlation exists. The following section presents an in-depth case study, shedding light on the results of this analysis.

    A Success Story: Real-World Impact of Digital Shelf Analytics

    Let’s dive into the journey of one of our clients – a prominent CPG brand specializing in the sale of baked goods and desserts. Through their experience, we will illustrate the transformative impact of our DataWeave Digital Shelf Analytics product suite.

    Over a period of one year, from August 2022 to July 2023, the brand leveraged several key modules of Digital Shelf Analytics for Amazon, including Share of Search, Share of Category, Availability, Ratings and Reviews, and Content Audit. Each of these digital shelf KPIs played a vital role in shaping the brand’s performance across various stages of the buyer’s journey.

    The buyer’s journey is typically delineated into three key stages:

    • Awareness: At this stage, shoppers peruse multiple product options presented on search and category listing pages, gaining an initial understanding of the available choices.
    • Consideration: Here, shoppers narrow down their selections and evaluate a handful of products, moving closer to a purchase decision.
    • Conversion: In this final stage, shoppers make their ultimate product choice and proceed to complete the purchase.

    Let’s now examine the data to understand how digital shelf KPIs helped drive tangible ROI on Amazon for the brand across the stages of the buyer journey.

    Stage 1: Raising Awareness

    Enhancing Share of Search and Share of Category can help brands boost product visibility and raise brand awareness. The following chart demonstrates the steady, incremental improvements in our client’s Share of Search and Share of Category (in the top 20 ranks of each listing page) throughout the analyzed period. These enhancements were achieved through various measures, including product sponsorship, content enhancement, price optimization, promotional initiatives, and more.

    This amplified Share of Search and Share of Category directly translates into improved product discoverability, as evident from the surge in impressions depicted in the chart below.

    Stage 2: All Things Considered

    In the consideration stage, shoppers make their product selections by clicking on items that meet their criteria, which may include factors like average rating, number of ratings, price, product title, and images. For brands, this underscores the importance of crafting meticulously detailed product content and accumulating a substantial number of ratings.

    The subsequent chart illustrates the year-long trend in both average ratings and the number of ratings, both of which have displayed steady improvement over time.

    The enhancements in the number of ratings and the average rating have a direct and positive impact on product consideration. This, in turn, has led to a noticeable year-over-year increase in page views, as indicated in the chart below.

    These improvements are likely to have also been influenced by the overall enhancement of content quality, which is detailed separately in the section below.

    Stage 3: Driving Decisions

    As buyers progress to the next stage, they reach the pivotal point of making a purchase decision. This decision is influenced by multiple factors, including product availability, content quality, and the quality of reviews, reflecting customer sentiment.

    Our client effectively harnessed our Availability insights, significantly reducing the likelihood of potential out-of-stock scenarios and enhancing replenishment rates, as highlighted in the chart below. The same chart also indicates improvements in content quality, measured by the degree to which the content on Amazon aligns with the brand’s ideal content standards.

    Below, you’ll find the year-over-year growth in conversion rates for the brand on Amazon. This metric stands as the ultimate measure of business impact, directly translating into increased revenue for brands.

    As the data uncovers, growth in key digital shelf KPIs cumulatively had a strong correlation with impressions, page views, and conversion rates.

    It is also important to note that the effect of each KPI cannot be viewed in isolation, since they are often interdependent. For example, improvement in content and availability could boost Share of Search. Accurate content could also influence more positive customer feedback. Brands need to consider optimizing digital shelf KPIs holistically to create sustained business impact.

    Impact on eCommerce Sales

    After the implementation of digital shelf analytics, the results spoke for themselves. Sales consistently outperformed the previous year’s records month after month. As shown in the chart below, the diligent application of DataWeave’s recommendations paved the way for an impressive 8.5% year-over-year increase in sales, leaving an indelible mark on the brand’s eCommerce success.

    From boosting product visibility to catapulting conversion rates, Digital Shelf Analytics serves as the key to unlocking unparalleled online success.

    While the success story detailed above does not establish a direct causation between Digital Shelf Analytics and sales revenue, there is undoubtedly a strong correlation. It’s evident that digital shelf KPIs play a pivotal role in optimizing a brand’s eCommerce performance across all stages of the buyer journey. Hence, for brands, it is vital that they collaborate with the right partner and harness digital shelf insights to fine-tune their eCommerce strategies and tactics.

    That said, the eCommerce landscape is in a constant state of flux, and there is still much to learn about how each digital shelf KPI influences brand performance in the online realm. With more data and an increasing number of brands embracing Digital Shelf Analytics, it’s only a matter of time before a direct causation is firmly established.

    Reach out to us today to know more about how your brand can leverage Digital Shelf Analytics to drive higher sales and market share in eCommerce.

  • Revolutionizing Fuel Pricing: How Fuel Retailers and Convenience Stores Can Gain a Winning Edge with DataWeave

    Revolutionizing Fuel Pricing: How Fuel Retailers and Convenience Stores Can Gain a Winning Edge with DataWeave

    Consider this scenario: A retailer establishes its fuel prices using pricing data that’s a few days old, only to subsequently discover that a nearby competitor is offering substantially lower prices. The result? Lost customers, decreased foot traffic, and diminished sales. This serves as a stark reality that retailers must confront and address today.

    In the fiercely competitive realm of retail, where every decision holds weight, maintaining a competitive edge is paramount. The fuel category, frequently underestimated, has the potential to significantly impact a retailer’s revenue stream. This challenge is not unique; retailers worldwide, particularly in North America, grapple with a common hurdle: mastering the intricate art of real-time fuel pricing.

    The Quest For Reliable, Real-Time Fuel Pricing Data

    For retailers, traditional methods for procuring and analyzing fuel price data have proven to be both expensive and error-prone, often relying on manual data collection or third-party data providers. These outdated approaches yield frustrating delays, inaccuracies, and missed opportunities. When it comes to obtaining timely fuel pricing intelligence, the majority of fuel retailers grapple with three central challenges:

    • Low Accuracy: Ensuring that fuel pricing information remains up-to-date, dependable, and actionable, even when sourced from complex web-based platforms.
    • Less Coverage: Acquiring comprehensive data that encompasses all of North America, spanning across retailers, convenience stores, fuel stations, and beyond.
    • High Cost: Effectively managing the substantial costs associated with acquiring and processing this vital information.

    DataWeave’s Fuel Pricing Intelligence Solution

    Comprehensive, accurate, and real-time fuel pricing intelligence can play a huge role in the profitability of retailers throughout North America. DataWeave takes the forefront in delivering this transformative Data-as-a-Service (DaaS) solution to some of the most prominent retailers in the region, including the top 20 fuel retail behemoths.

    With a rich and extensive history spanning over a decade in the realm of competitive intelligence, DataWeave boasts an impressive track record of empowering well-informed decision-making in retail. We leverage state-of-the-art technology to bring an unparalleled level of accuracy, timeliness, and coverage to fuel pricing intelligence.

    The following are some compelling advantages offered by our solution:

    Accurate and Real-Time First Party Data

    We deliver retailers an unparalleled advantage through real-time, first-party fuel price data. Our data originates directly from the retailer’s own channels, encompassing websites and mobile apps, rendering it the industry’s foremost and most reliable source.

    Imagine having access to fuel pricing information that updates as frequently as every 30 minutes. This rapid update cadence guarantees that you, as a retailer, constantly possess the latest pricing insights at your fingertips, empowering you to respond swiftly to market fluctuations and competitor manoeuvres. Our comprehensive data spans a wide spectrum of fuel types, including:

    • Gasoline: Be it regular, mid-grade, super, premium, ethanol-free, ethanol blends, methanol blends, or reformulated gasoline, we have got you covered.
    • Diesel: Our data encompasses biodiesel, biodiesel off-road, biodiesel blends, biodiesel ultra-low sulfur (ULS), diesel ultra-low sulfur (ULS), diesel off-road, standard diesel, and premium diesel.

    Armed with our real-time, first-party data, you can make pricing decisions with unwavering confidence, secure in the knowledge that you possess access to the most current, authoritative, and extensive fuel pricing intelligence in North America.

    The data points we capture directly from relevant web sources include: gas station postal code, store name and code, location, city, state, ZIP code, fuel type, competitor name, regular price, member price (if available), time and date of data capture, and more.

    Click here if you wish to access a sample report of our fuel pricing data.

    Unrivaled Geographical Coverage

    Our extensive coverage of fuel data spans over 30,000 ZIP codes and encompasses the top 100 retailers across the western, mid-western, and eastern regions of the United States.

    Retailers benefit from the flexibility to configure and tailor the solution to their precise needs, whether it involves adding more locations or selectively acquiring specific segments of the data. This far-reaching coverage guarantees that retailers, whether situated in bustling urban centers or remote areas, can readily access the essential data required to maintain their competitive edge.

    Moreover, if you currently source your fuel pricing data from alternative providers, our solution seamlessly integrates, amplifies, and complements your existing array of data sources, ensuring a harmonious and unified approach to data acquisition.

    Optimization of Dynamic Pricing Strategies

    In the world of retail, the importance of timing cannot be overstated. Even a mere difference of a few cents can translate into millions of dollars in revenue impact. With DataWeave, retailers gain the capability to make data-driven decisions that provide them with a competitive edge around the clock, every single day.

    Our platform empowers you to unearth margin gaps by pinpointing opportunities to raise prices while maintaining your competitive pricing position. It also identifies instances where you may be substantially overpriced, prompting necessary price adjustments to ensure competitiveness within the market. All these valuable insights are available at a hyperlocal level, facilitating pricing efficiency and optimization across your various regions of coverage. Equipped with this real-time data, you can swiftly adapt to ever-changing market conditions.

    Furthermore, our comprehensive competitive data seamlessly integrates into your existing pricing systems through APIs, facilitating quick and informed pricing actions based on robust data.

    Reliable and Customer-First Tech Platform

    Our platform boasts a remarkable level of sophistication when it comes to data aggregation, normalization, visualization, and integration capabilities. It stands as a massively scalable system with the capacity to aggregate billions of data points daily, spanning thousands of web sources. This includes the intricate handling of sources like mobile apps and websites known for frequently altering their site structures, among others.

    What truly sets us apart is our proficiency in addressing these challenges through a blend of human expertise and large-scale machine learning. Additionally, our commitment to delivering unmatched service extends to round-the-clock, 24/7 support. This comprehensive approach makes our fuel pricing intelligence solution not only effective but also cost-efficient in meeting your fuel data requirements.

    We also provide a variety of options for you to consume our data, which includes receiving our reports via email, SFTP, S3 buckets, data lakes like Snowflake, and APIs.

    Enhance your Fuel Pricing Strategies with DataWeave

    In the ever-competitive world of retail, staying ahead is not just a goal; it’s a necessity. The fuel pricing landscape, often overlooked, holds immense power to impact a retailer’s profitability. DataWeave’s real-time, comprehensive, and accurate fuel pricing intelligence solution is the key to securing this advantage. Retailers and convenience stores now have a powerful platform at their disposal, offering unparalleled precision, comprehensive coverage, and the agility needed to navigate this landscape.

    Join the ranks of industry leaders who have already harnessed the potential of DataWeave. Reach out to us today to redefine your approach to fuel pricing and propel your business to new heights!

  • 5 Ways DataWeave Helps Brands Drive Growth With Amazon Ads

    5 Ways DataWeave Helps Brands Drive Growth With Amazon Ads

    Consumers are discovering and trialing new eCommerce marketplaces, brands and products at a faster rate than ever before, given the vast amount of choices encountered browsing for products online. A recent analysis shows how events like Amazon Prime Day, Black Friday, and Cyber Monday are especially fruitful for new-to-brand customer advertising, encouraging B2C marketers to increase their digital advertising spend to fuel product discovery, sales and market share for their brands.

    Amazon advertisers grow market share and brand loyalty with ecommerce intelligence
    DataWeave joins Amazon Advertising partner network

    The majority of eCommerce consumers are discovering products via relevant keywords attributable to their needs, with most clicks happening on page one results for the first few products listed. Simplifying the digital shopping experience is critical for brands to be in the consideration set for the majority of consumers who won’t venture past page one results. 

    An internal analysis conducted shows getting a product to page one on retailer websites can improve sales by as much as 50 percent, but figuring out the right levers to pull to get there organically—without paid advertising—is a real challenge, especially given fast-changing algorithms. While more than half of all retail related online browsing sessions are “organic”, sometimes brands need to boost their product visibility by investing in sponsored (paid) opportunities to improve a product’s rank.

    Data analytics can equip brands with intelligence to help them decide when, where, and how to make digital advertising investments profitably, while simultaneously acting on insights that help drive organic growth. Considering a majority of U.S. consumers begin their product discovery on marketplaces like Amazon, it makes sense for brands to prioritize digital advertising opportunities with Amazon.

    Maximize Return on Ad Spend (ROAS) with Amazon Ads

    Brands use Amazon Ads to drive brand awareness, acquire new customers, drive sales and gain market share, with the goal of furthering their marketing return on investment. Top performing advertisers average 40 percent greater year-on-year (YoY) sales growth, 50 percent greater YoY growth in customer product page viewership on Amazon, and 30 percent higher returns on ad spend (ROAS) with Amazon Ads, according to a recent analysis. Sponsored Products, Sponsored Brands, Amazon DSP and Sponsored Display are among the types of Amazon Ads options cited that produce maximum return.

    Ensuring your product listings appear at the top of page one results on Amazon for the most relevant discovery keywords is therefore the most important determinant for maximizing ROAS. DataWeave has become a vetted partner and measurement provider in the Amazon Advertising Partner Network, with the goal of supporting brands to optimize digital advertising campaigns by providing visibility to Digital Shelf Analytics (DSA) key performance indicators (KPIs), like Share of Search, Pricing and Product Availability, Content Audits, Ratings and Reviews, and Sales Performance and Market Share.

    Below is a summary of how our Digital Shelf solutions, in partnership with Amazon Ads, can improve the performance of your Amazon Ads campaigns

    1. Keyword Recommendations Improve Share of Search

    With the DataWeave Share of Search solution, brands can monitor their placement of both organic and paid discovery keywords relative to their competition. Once your keywords are determined, you are also provided a weighted Share of Search score that helps measure how well each keyword performs relative to product discoverability. Below is an example of insights you’d gain.

    Share of Keyword Search

    Brands can provide their own list of keywords to monitor, or through our Amazon Ads collaborative solution, learn which keywords are the “best” for them to measure in the realm of Amazon. Performance results are based on data that shows which keywords consumers are actually using when browsing online alongside other keywords brands request to measure. Users are able to see exactly which keywords are most popular, competitive (and even unexpected), and relevant at an Amazon Standard Identification Number (ASIN) level of granularity. 

    We can also estimate the degree of relevance and estimated traffic for the recommended keywords. Brands can then use these insights to adjust campaign strategies based on these parameters, which can boost product discoverability and rank visibility. A brand could assume people find its products by brand name, yet traffic insights may reveal a majority of people look for a generic product type before they end up buying that particular brand. 

    2. Content Audits Increase Discovery Relevancy Scores

    Strong product content is critical to succeeding on Amazon. Thorough, accurate, and descriptive content leads to better click through rates (CTR), conversion rates, more positive reviews, and fewer returns, which results in increased discoverability. DataWeave’s Content Audit solution reviews existing copy and images on a per-attribute basis to highlight any gaps essential to improving visibility, as seen in the example below.

    Content Analysis

    To further growth, it is equally as important that your product content aligns with your advertising strategy. With Amazon Ads partner add-on, our solution can also audit your content to measure how effectively you are incorporating Amazon Ads keywords into your product content to enhance discovery relevancy.

    3. Discover More Opportunities with Pricing and Product Availability Insights

    Quality content and keyword updates will only get you so far if your products are not consistently available and priced competitively. With DataWeave’s Pricing and Promotions and Product Availability modules, advertisers can monitor their selling prices and availability trends alongside their competitors to uncover more opportunities to incorporate into advertising campaigns, as seen in the Pricing and Promotions dashboard example below.

    Promotion Analysis

    Additionally, product targeting recommendations can be utilized to target a competitor’s ASIN that may be overpriced or that is having issues staying in stock. Alternatively, broaden your strategy to target specific brands, complementary products, or category listing pages.

    You can also create alerts on your own products to monitor when items are low on inventory or out of stock to ensure key products are consistently available when customers are shopping.

    4. Leverage Ratings and Reviews to Increase Conversion

    Product ratings and reviews are also a critical component to running a successful Amazon Ads campaign. A large number of reviews and a positive star rating will provide customers with the confidence to purchase, resulting in higher conversion rates. Conversely, negative feedback can have a detrimental impact, resulting in lost sales and wasted ad spend. DataWeave’s Ratings and Reviews module can help you monitor your reviews and extract attribute-level insights on your products. This information can then be utilized to further optimize your advertising strategy.

    If you see consistent feedback in your reviews on aspects of a product not meeting customer expectations, address them in your product content to prevent potential misplaced expectations. Alternatively, if customer reviews are raving about certain product features, ensure these are promoted and relevant keywords are populated throughout your descriptions and feature bullets. Below is an example of insights seen within the DSA Ratings & Reviews dashboard.

    Ratings and Reviews

    5. Correlate Digital Shelf KPIs to Sales Performance and Market Share

    The newest DSA module, Sales Performance and Market Share, provides SKU, sub-category, and brand-level sales and market share estimates on Amazon for brands and their competitors, via customer defined taxonomies, to easily benchmark performance results.

    This data can also be correlated with other Digital Shelf KPIs, like Content Audit and Product Availability, giving brands an easy way to check the effect of attribute changes and how they impact sales and market share. Similarly, brands can see how search rank, both organic and sponsored, affects sales and market share estimates.

    Understanding the correlation between your advertising campaigns and your Digital Shelf brand visibility will help you identify which areas to prioritize to drive sales and win more market share.

    Digital Shelf Insights Help Brands Win with Amazon Ads

    The need for access to flexible, actionable eCommerce insights is growing exponentially as a way to help brands drive growth, increase their Share of Voice, and to gain a competitive edge. As a result, more global brands are seeking Digital Shelf Analytics for access to near real-time marketplace changes and to develop data-driven growth strategies that leverage pricing, merchandising, and competitive insights at scale.

    By monitoring, measuring and analyzing key performance indicators (KPIs) like Sales Performance and Market Share, Share of Search, Content Audits, Product Availability, Pricing and Promotions and Ratings and Reviews alongside competitors, brands will know what actions to take to boost brand visibility, customer satisfaction, and online sales. 

    DataWeave’s acceptance into the Amazon Advertising Partner Network enables Amazon advertisers to effectively build their Amazon growth strategies and determine systems that enable faster and smarter advertising and marketing decision-making to optimize product discoverability and overall results.

    Connect with us now to learn how we can scale with your brand’s analytical needs, or for access to more details regarding our Amazon Ads Partnership or Digital Shelf solutions.

    UPDATED: Read the full press release here

  • U.S. Prime Day Deals 2022: Promotion Intelligence First Look

    U.S. Prime Day Deals 2022: Promotion Intelligence First Look

    As inflation hits another 40-year high at 9.1 percent, U.S. consumers geared up for their first sign of hope and relief in the form of anticipated discount buys – 2022 Amazon Prime Days, or so we thought. While Prime Days have grown to become a promotional period almost as important as Black Friday to digital shoppers, the combination of economic uncertainty, inflationary pressures, and supply chain challenges seemed to alter the discount strategy expected given activity seen during 2021 Prime Days.

    Our analyst team has been hard at work aiming to provide a ‘first look’ at 2022 Prime Day Promotional Insights, tracking discounts offered across 46,000+ SKUs within key categories like Electronics, Clothing, Health & Beauty and Home, on seven major retailer websites – Amazon, Target, Best Buy, Sephora, Ulta, Lowe’s and Home Depot. Our analysis compares prices seen during Amazon Prime Day 2022 on July 12th, to pre-Prime Day maximum value prices seen in the ten days leading up to Prime Days, to determine the average change in discounts offered during the promotional period. Below is a summary of our findings.

    Competitive Promotions Give Amazon a Run for their Money

    Amazon offered the greatest average discount enhancements for Electronics at 5.6 percent followed by Health & Beauty items at 5.1 percent, and Home products at 4.2 percent versus pre-Prime Day discounts seen across the categories considered within our analysis. The only category reviewed where average discounts were greater on a competitor’s website was on Target.com within the Clothing category. As seen below, Clothing on Target.com average discounts were 6.8 percent greater than pre-Prime Day offers, which was 2.6 percent higher than the average discounts offered for Clothing on Amazon.

    Target Capitalizes on Growth Opportunity in Clothing Category

    Diving deeper into the details of where Target won within the Clothing category, you can see a majority of their promotional activity took place within Women’s Accessories where discounts offered were 18.5 percent greater than those seen pre-Prime Day 2022, which was almost 15 percent greater than the discount enhancements seen on Amazon for Women’s Accessories. In fact, Women’s Shoes and Sneakers were the only two categories where the average discounts offered were greater on Amazon than on Target.com.

    Overall, the discounts offered on Target.com within the Clothing category were primarily concentrated within items priced $40 and lower, but what was most interesting is that within the $10 and under price bucket, Target offered average discounts of over 11 percent whereas Amazon increased prices for these items on average by over 9 percent.

    While most of the Clothing available on both Amazon and Target.com during Prime Days 2022 were offered without a price change, the greatest discount percentages tracked were within the range of 10-25 percent off on Amazon whereas Target chose to offer the bulk of their promotions at 25 percent off an up.

    Strategic Promotional Strategies Defined at the Electronics Subcategory Level

    When it comes to the Electronics category on Prime Day, the big question is always who will win the battle of the brands. Below shows the difference in average pricing and promotions discounts offered between products manufactured by Samsung versus Apple across each retailer platform, noting discounts were almost 3 percent greater on average for Apple versus Samsung products on Amazon, and Apple discounts were almost 5 percent greater on Amazon versus than those seen on Target.com.

    Amazon wasn’t going all in on Apple however, as we saw ‘Alexa’ devices (Amazon products) available on Best Buy and Target websites also, but the discounts were almost 4 percent greater on Amazon versus Target and over 7 percent greater than the discounts seen on BestBuy.com.

    While the average discounts offered within the Electronics category were greatest on Amazon (5.6 percent) versus Best Buy (3.9 percent) and Target (3.4 percent) as noted within the first chart of this blog and across brands and technologies considered above, the discounts offered on Amazon were strategically focused between 10-25 percent as seen below.

    Amazon’s Electronics promotions were also targeted at smaller price points, items priced between $20-500, whereas Best Buy and Target offered greater promotions for electronics priced $500 and up than Amazon.

    Below is a snapshot of price buckets tracked for Electronics available on BestBuy.com, highlighting where most of the promotional activity was targeted at products priced $50 and up during Prime Days 2022, with discounts ranging from 10 percent up to greater than 25 percent greater than pre-Prime day prices.

    The standout categories were TVs on Target.com with discounts averaging nearly 12 percent greater than those seen pre-Prime day, and smartphones on BestBuy.com with discounts averaging just over 11 percent greater than those seen pre-Prime Day. The category with the greatest average discount enhancements seen on Amazon during Prime Days 2022 was for Wireless Headphones with an average discount of 8.7 percent.

    Home is Where Amazon’s Heart Was on Prime Day

    Amazon dominated offers within the Home categories, especially for products within mid ($40-100) and higher price ranges (items priced $200-500), with the bulk of the discounts offered between 10-25 percent. There was little to no promotional activity seen across all price points on Lowe’s or Home Depot’s websites within the categories we tracked, and most other competitive offers on Home products were seen on BestBuy.com for products priced from $50-500. Even a subcategory like Tools offered deeper average discounts on Amazon (4.7 percent) than discounts seen on HomeDepot.com (1.1 percent) or Lowes.com (0 percent).

    For Large Appliances, Amazon was the only retailer to off any significant discount across each major subcategory with the greatest average discount being on Ovens at 6 percent, followed by Refrigerators at 4 percent. One caveat with this category, when we reviewed Large Appliance prices two weeks prior to Prime Days, we saw average price increases around 16.7 percent occurring on Amazon.

    During Prime Days 2022 however, Amazon also offered top average discounts for small appliances, except for on Instant Pots which appeared to have greater average discounts on Target.com (5.9 percent versus 4.2 percent on Amazon), and Vacuum Cleaners which appeared to have the best promotion of appliances small and large at 13.8 percent average discount on BestBuy.com. Another subcategory deeply discounted on BestBuy.com was weighted blankets, which averaged discounts around 18.5 percent versus Amazon’s average discount at only 6.2 percent.

    Health & Beauty Retailer Pricing Strategies Revealed

    Given the importance Health & Beauty Brands placed on Prime Day sales last year, we had anticipated to see more offers, especially within pure-play beauty retail channels, than we did for this booming category.

    Amazon drove most of the Health & Beauty offers seen averaging 5.1% discounts versus other retailers only offering less than 1% on average, but discounts were aimed at a targeted group of SKUs on Amazon, bringing the average discount lower overall. Most of the promotions offered on Amazon fell within mid-range price points ($20-50) and were discounted between 10-25 percent versus pre-Prime Day prices.

    Target.com offered the most comparable discounts to Amazon for Health & Beauty products on average, but their strategy primarily focused on items within the $20 and lower price range with discounts ranging primarily between 10-25 percent.

    More 2022 Prime Day Insights Coming Soon

    We know the significance visibility to critical pricing and promotional insights play in enabling retailers and brands to offer the right discounts to stay competitive, especially during promotional periods like Prime Days. While this blog is intended to provide a ‘sneak peek’ into 2022 Prime Day insights for the U.S. market, we will be providing more extensive, global coverage and will proactively share new insights with the marketplace as they become available throughout the month of July.

    Be sure to also check out our Press page for access to the latest media coverage on Prime Day insights and more. Don’t hesitate to reach out to our team if there is any particular category you are interested in seeing in more detail, or for access to more information on our Commerce Intelligence and Digital Shelf solutions.

  • 7 Key Metrics that QSRs want (but may not get) from Food Delivery Apps

    7 Key Metrics that QSRs want (but may not get) from Food Delivery Apps

    The Quick Service Restaurant market is projected to be valued at $691 billion by 2022. As the QSR industry grows and the market becomes even more competitive, restaurant chains continuously seek ways to increase sales via food aggregators to market their business. To improve ROI and sales, having data and insights into key metrics could help QSRs to boost their success rate.

    QSRs would like to know how they stack up against their competition regarding discoverability on cluttered food aggregator apps. Restaurants want to know the gaps in their product assortment to understand what drives customers to their competitors. Getting insights into delivery time and competitors’ delivery fees will help QSR improve delivery ETAs and optimize fees. They can also set competitive pricing with insights into their competitors’ pricing. In addition, they can use data to optimize their ad spending on food apps and improve marketing ROI.

    In this blog, we will discuss the relationship between QSRs and food aggregators and how getting data about key metrics from these food delivery platforms can help QSRs scale their revenue. 

    Data: The Key Ingredient to increasing sales

    According to Statista, online food ordering revenue is expected to grow at a robust CAGR of 10.39% between 2021 and 2025. Food Aggregators apps like Uber Eats, DoorDash, and GrubHub offer convenient meal delivery options from various QSRs within a single app. Food aggregators provide a multitude of benefits for QSRs. They give access to a huge customer base, quick delivery, and an easy entry into quick commerce, helping QSRs increase visibility. Although QSRs rely on food aggregator platforms for hassle-free ordering, tracking, and delivery, they can’t always rely on them to share critical data that could help them optimize their operations & increase sales. 

    Online food ordering revenue
    Online food ordering revenue

    1. Data on Product Assortment

    QSRs need assortment insights to understand their competitor’s menu assortment. Assortment analytics plays a crucial role in ensuring that QSRs aren’t losing sales because their competitors are offering cuisines and dishes that they aren’t. Understanding gaps in menus helps QSRs to better plan their menu. However, food aggregator apps can’t share competitors’ assortment data with QSRs for a multitude of reasons, guidelines, and privacy laws. Thankfully, at DataWeave, our QSR intelligence solution can! We help restaurants improve their assortment by sharing insights into the dishes and cuisines their competitors’ have on display.

    Menu Assortment
    Menu Assortment

    2. Data on QSR Discoverability

    QSRs would love to know how to increase discoverability on food aggregators, as it will help them to appear ahead in search results and beat the competition. Improving visibility on these apps directly impacts sales and drives more orders for restaurants. Some aggregators offer discoverability information but give it on demand, usually after 20-30 days, making it irrelevant due to the enormous time gap. They also don’t provide information about the change in the discoverability of your competition. All these data points are so critical, and understandably so, Food Apps can’t share this level of information with restaurants. However, DataWeave’s QSR Intelligence solution can! It provides real-time discoverability insights into your restaurant and competitor’s visibility so that the data is actionable, and QSRs can use insights to improve visibility

    Read how DataWeave’s QSR Intelligence helped an American QSR Chain and how their ranking on search results page on Ube rEats, DoorDash & Grubhub impacted outlet discoverability & sales!

    3. Data on Pricing & Promotions

    Pricing a QSR’s menu is tricky. If you price too high, you’ll turn off new customers. If you price too low, you’ll cut margins & may even come off as low-qualify. Customer Price Perception is greatly influenced by the Price-Quality relationship. To add to this, restaurants are often up against stiff competition from restaurants with similar cuisine offerings so it’s critical that prices are competitive. Understanding competitor pricing doesn’t imply that you have to beat their prices. You can compensate for any price differences by offering higher quality cuisines, better customer service, and quicker delivery. Once again, food apps can’t share competitors’ pricing data with QSRs. But DataWeave’s QSR & Pricing Intelligence solution can! QSRs can use these insights to drive more revenue & margins by pricing their menu right.

    4. Data on Delivery Time

    QSRs must be able to deliver hot meals, in a timely manner to customers because customers want to quickly dig into the delicious food they ordered. Quicker deliveries within the ETA will also help earn the trust and loyalty of customers. However, food aggregators don’t share information on the delivery times with restaurants – not their own delivery time or their competitors. DataWeave can help QSRs to understand their peak hours and optimize their service to ensure quick ETAs. They can also get detailed insights into competitors’ delivery times to make sure they’re competitive. This is important because customers will often pick restaurants with quicker ETAs.


    Read how DataWeave’s QSR Intelligence helped an American QSR Chain understand the correlation between delivery time & sales volumes

    Delivery time trend by urbanity
    Delivery time trend by urbanity

    5. Data on Delivery Fee

    As a thumb rule, customers will always compare delivery fees across apps. They’re conscious of delivery dollars included in their bill and often choose a restaurant with lesser delivery fees. This makes it even more critical for restaurants to understand how they stack up against their competitors. Understanding competitors’ delivery fees could potentially help QSRs to optimize their rates. And once again, food aggregators can’t share information on competitors’ delivery fees with restaurants. However, DataWeave’s QSR Intelligence can provide all delivery-related insights – be it Delivery etas or fees. 

    Delivery fee trend by urbanity
    Delivery fee trend by urbanity

    6. Data on Ad Performance & ROI

    Getting ad analytics will help QSRs better manage their budgets & increase the ROI on their Ad spends. For example, wouldn’t it be great if QSRs were able to understand which ad formats or promotions led to the most sales? Or which carousal ads had the most visibility in key zip codes where your QSR is expected to do maximum business? Or even insights into a competitor’s ads and promotions on food apps. Knowing this information will help restaurants spend sensibly when buying media on Food Apps & get the most bang for their advertising buck. Food apps do provide standard ad analytics – a number of clicks, CTR, and so on, but for more complex, insightful & actionable insights, there’s DataWeave’s QSR Intelligence

    Read how DataWeave’s QSR Intelligence helped an American QSR Chain understand the ROI delivered on ad spends across Food Delivery apps.

    Insightful & actionable insights for QSR Chains
    Insightful & actionable insights for QSR Chains
    Insightful & actionable insights for QSR Chains
    Insightful & actionable insights for QSR Chains

    7. Data on Outlet Availability / Availability Audit

    To avoid lost sales, being available & “open for business” on Food Apps during peak lunch & dinner hours is critical. Also on weekends, when order volumes are usually high. Sometimes because of technical glitches, QSR outlets appear unavailable on Food Apps. A glitch like that can lead to lost business, and the longer the glitch stays undiscovered, the greater the impact on revenue. While Food Aggregators do their best to make sure all QSRs are up and running on their app, using DataWeave’s QSR Intelligence, restaurants can now do an outlet audit to make sure that’s the case. With just a mere 2.8% unavailability, we saw a 28% drop in the sales for one of our QSR customers! That’s how critical Availability insights are. 

    Conclusion

    Analyzing and optimizing sales, delivery, discoverability, availability & customer data is one of the fastest ways to help grow your QSRs revenue. However, the biggest challenge QSRs face is that it isn’t always easy to get this information. With DataWeave’s QSR Intelligence now some of that data is a little more accessible as we discussed in this blog. And additionally, here are the 7 Tricks we recommend QSRs to use to win on Food Apps

  • The Future of eCommerce is Social: Demystifying the Social Commerce Revolution

    The Future of eCommerce is Social: Demystifying the Social Commerce Revolution

    Social commerce is the selling of goods and services within a social media platform. Brands use social platforms such as Instagram, Facebook, Snapchat, and Twitter to promote and sell products. These platforms have become an integral part of consumers’ everyday life because they continue to engage users with relatable content, making them scroll their feeds for hours. 

    The Social Commerce model capitalizes on this high user engagement & moves social media beyond its traditional role in the top-of-the-funnel marketing process by encouraging users to shop without leaving their preferred apps. According to the Social Media Investment Report, 91% of executives agree that social commerce is driving an increasing portion of their marketing revenue, and 85% report that social data will be a primary source of business intelligence.

    Let’s talk a little bit about why brands should consider selling via social media platforms:

    Social Commerce vs. eCommerce vs. QCommerce

    While they may fall under the same umbrella of online selling, social commerce, quick commerce, and eCommerce are three very different concepts

    • eCommerce refers to online shopping via a (retailer or brand) website or app. Customers can access these platforms via desktop or mobile devices. However, the sales funnel generally looks the same. These brands and retailers use top-of-the-funnel tactics like social media content, digital ads, and other marketing strategies to encourage customers to visit the online store. There are three main types of eCommerce businesses: Business-to-Business (Alibaba, Amazon Business, eWorldTrade), Business-to-Consumer (websites such as Amazon, Rakuten, and Zalando), and Consumer-to-Consumer (platforms such as eBay & Etsy).
    • Quick Commerce (or QCommerce) refers to eCommerce businesses that deliver goods within a couple of hours or even minutes. Although it’s sometimes used interchangeably with on-demand delivery or instant commerce, the idea of quick commerce has been around in the food industry for ages now. It has been recently ushered into the mainstream by evolving consumer preferences for quicker delivery of groceries and FMCG goods.
    • Social commerce brings the store to the customer rather than redirecting customers to an online store. It removes unnecessary steps and simplifies the buying process by letting the customer checkout directly through social media platforms, creating a frictionless buying journey for the customer. Additionally, social media platforms are mobile-friendly, a huge benefit for brands because increasingly more and more customers are accessing the internet through mobile devices.
    Social Commerce
    Social Commerce

    Rise of Social Commerce

    First used in 2005 by Yahoo!, ‘social commerce’ refers to collaborative shopping tools such as user ratings, shared pick lists, and user-generated content. Social media networks snowballed throughout the 2000s and 2010s, alongside a general increase in eCommerce, leading customers and merchants to quickly recognize the benefits of buying and selling through social media networks. Social media platforms have since evolved from merely a showcase tool for brands. They now serve as virtual storefronts and extensions of a company’s website or brick and mortar stores, capable of handling the buying experience.

    Top Social Commerce Platforms

    Social media platforms aim to keep visitors engaged on their platforms for as long as possible. Increased time in-app or on-site maximizes their opportunity to serve ads, a primary source of revenue generation. Social media platforms have millions of active users and they have a great power to help companies and individuals build their brands, interact with consumers, and support after-sales. Here are the top social commerce platforms:

    • Facebook

    Facebook introduced Facebook Shops to capitalize on the commercial opportunity by allowing vendors to advertise and sell directly through the platform. Facebook integrates social commerce with shopping, allowing users to purchase products smoothly. Facebook shops offer a smooth user experience where users can review products and get recommendations from trusted acquaintances. Customers can directly interact with the merchant’s customer service department post-purchase. 

    • Instagram

    60% of people discover new products on Instagram. Owned by Facebook, Instagram facilitates in-app shopping and handles the entire transactions within the app itself. Users scrolling on Instagram often wants to follow trends and replicate the looks of their role models or favorite influencers. By offering purchasing options in the app, Instagram benefits from the platform’s rich visual imagery and videos, allowing businesses to sell an idea rather than the traditional process of selling a product. 

    • TikTok

    Shopify partnered with TikTok to introduce shopping and drive sales through the younger and seemingly ever-expanding TikTok audience. With TikTok for Business Ads Manager, brands and merchants can create in-feed video-based content depending on their product offering. This partnership allows Shopify merchants to expand to the TikTok audience.

    • Snapchat

    Snapchat has recently launched Brand profiles, a feature that allows users to scroll through a merchant’s products and buy them in-app. This new experience is powered by Shopify too. Merchants can create Brand Profiles or Native Stores that allow users to purchase products from the app. 

    Pinterest users are there for Shopping Inspiration
    Pinterest users are there for Shopping Inspiration
    • Pinterest

    Pinterest is also an image-based platform where users create boards of their favorite wedding accessories, home decor, fashion trends, etc. Pinterest doesn’t specifically offer social commerce for the global audience. Rather, it allows business accounts to create ‘Product Pins’ that are displayed in the brand’s Pinterest shop. Only U.S. customers can purchase within the app. Users from other countries are redirected to the eCommerce site to complete the sale. We have added Pinterest to this list because 89% of Pinterest users are there for shopping inspiration.

    Pinterest is an image-based platform where users create boards
    Pinterest is an image-based platform where users create boards

    Why Should Brands Care About Social Commerce

    • To enhance social media presence and brand awareness

    If your target demographic is in the 18-to-34 age range, they’re already on social media and waiting to shop while they scroll. According to Sprout Social, over 68% of consumers have already purchased directly from social media and nearly all (98%) consumers plan to make at least one purchase through social or influencer commerce this year. You can enhance brand awareness by selling on social media platforms. Influencer marketing is an amazing way to build brand awareness since customers are now seeking authenticity from micro-influencers rather than big-name celebrities. 

    • To generate social proof

    90% of online shoppers say that they read online reviews before making an online purchase. Whether it’s an automated follow-up email or a message through the social media platform, ask for a review after your product has been delivered to the customer. You can also offer incentives like a contest to encourage previous customers to weigh in and share their experiences. These steps will allow you to collect social proof since it’s vital to build a positive reputation online. You can also ask customers to create small product review videos that you can share on your social feeds in creative ways. You can also post user-generated content, create a carousel of positive comments, or host a live video with happy customers.

    Social Proof
    Social Proof
    • To simplify the buying process for consumers

    Traditional eCommerce involves several steps. It starts with displaying ads on social media platforms and customers being redirected to the business website for completing the transaction. To complete the transaction, customers also have to create an account or manually fill in the credit card details and delivery address. On the other hand, social is only a three-step process — find, click and buy. 

    Counterfeit Products
    Counterfeit Products

    Conclusion

    While social commerce is proliferating, it also has a few setbacks like the rise of counterfeit products. Counterfeiting has expanded into social media and has become an under-reported but vital hub for counterfeiters. A counterfeit detection solution can help brands and merchants identify & remove fake and unauthorized products. Technologies like image recognition can help in counterfeit detection by capturing fake logos and discrepancies. Removing counterfeit products will help brands safeguard customer loyalty and prevent fake products from harming your bottom line. 

    Here’s how DataWeave helped Classic Accessories, a leading manufacturer of high-quality furnishings & accessories identify counterfeit products across multiple retail marketplace websites eliminating 22 hours of time spent per week conducting manual audits – read the case study here

    Are you a brand or a retailer worried about counterfeits? Sign up for a demo with our team to know how we can help you track, identify and eliminate fakes! 

  • What is Customer Price Perception  and why it is important

    What is Customer Price Perception and why it is important

    Finding the right price often requires a trade-off between margin and price perception. Brands may want to defeat competitors’ prices on all their products, but that can often lead to losses because sales directly link to price perception. Instead of trying to stay competitive across the board on all products, brands must identify key value categories (KVCs) and key-value items (KPIs) whose prices buyers tend to remember and price those products competitively. In this scenario, they can make up for lowered prices on key products by fixing higher prices on other products. 

    Consumers’ perception of price fairness largely determines their experience with a brand. Brands selling online can often have a disconnect between their prices and what customers expect their prices to be. However, that does not mean spiraling downwards by getting trapped in discounting cycles and heavy promotions that can harm your bottom line. Instead, brands require real-time monitoring across thousands of stock-keeping units (SKUs) to identify key categories and items they need to price with care. In this blog, you’ll learn about price perception and the factors that influence it. 

    What is Price Perception?

    Price perception is the perceived worth of a product or service in the consumer’s mind. It is one of the leading variables in the consumer’s buying process. Buyers are unaware of the true cost of production for the products they buy. Instead, they make buying decisions based on an internal feeling about how much certain products are worth and which brand offers them the best value. To offer competitive prices and yet obtain a higher price for products, brands often pursue marketing strategies to improve the price perception of their brand and products.

    Price Perception
    Price Perception

    However, brands should not fall into the trap of assuming that price perception is a competitor’s price index. It’s not about offering the lowest price on certain SKUs. Not every brand strives to offer the lowest prices. Some brands take a slightly different approach to ensure the right value for their products. For example, take a look at Trader Joe’s, a grocery chain that has never claimed low costs. They’ve always taken a holistic approach to their pricing and customers to build a loyal following. And it worked well for them. Trader Joe’s can boast one of a high-value perception score, despite not having rock-bottom prices. 

    Marketplaces such as Walmart and Amazon may not have the best prices on every item. Still, customer perception is that they will have the lowest prices and will often shift the share of sales towards such platforms over businesses that offer the same or even lower prices. 

    Some things to consider:

    • What do your customers think of your brand?
    • What are the key factors that are driving your customers’ price perceptions?
    • Is your product mix properly aligned with your brand perception?
    • Are you communicating the most important and relevant information to your customers?
    • Is your message being received and understood?
    • Who do your customers see as your competitors, and why?

    Also Read: 11 Reasons why your eCommerce Business is fail 

    What is Price Positioning?

    Price positioning is pricing products or services within a certain price range. It indicates where certain services or products lie in relation to competitors’ pricing and in the mind of different customers. A brand’s price positioning has a huge impact on whether the products are seen as priced low or not. The following is a great way to understand the price-value matrix:

    Price Positioning
    Price Positioning

    Your brand’s position in this matrix will depend on your pricing objectives, competition, and customer loyalty. Price positioning helps the marketing and operating teams understand customers’ perceptions of your brand and convince customers to buy your products. Brands need a holistic approach toward setting prices for their products in order to drive conversions through intelligent pricing and competitive insights. 

    Factors that influence Price Perception

    Price-Quality Relationship

    Price is often an indicator of product quality. The general rule is that the higher-priced products are perceived to have better quality, implying that brands should consider a rational quality-price relationship in their pricing or promo strategy. For example, it might not be best practice to have similar prices for both good and low-quality products because customers will perceive low-quality products as overpriced and might not purchase from you.

    Price-Consciousness

    Customers aren’t price conscious about every product. Instead, they are only price conscious about certain products under the best price guarantee or BGP. For instance, if buyers find your BGP products more expensive than your competitors, the cheaper products in your assortment will still be perceived as expensive. 

    Value-Consciousness

    During markdown periods, ensure that you are not undermining the efforts to shape and maintain price perception by offering extreme or complex discounts. In an attempt to clear stocks, promotions simply confuse the shopping experience for customers and further deteriorate trust in your brand. Your promotional offers should keep price perception during the holiday season or clearance sales by offering a simplified promotional program. Start by understanding which price mechanics and SKUs work best for your target customer segment. You should also reduce over-communication on hero deals else buyers will assume that you incorrectly price products during new seasonal launches. 

    Prestige Sensitivity

    Gerald Zaltman, a Harvard professor, argues that 95% of all purchasing decisions are subconscious. Luxury brands are a great example of how psychology directly links to price perception. Customers buy premium or luxury products to demonstrate their social status. In this scenario, buyers don’t hesitate to buy expensive products from certain brands even if they are explicitly overpriced. Thus, brands selling premium products will have to ensure pricing is coherent with buyers’ expectations. 

    Every customer wants to know they’re getting the best value. They use the highest and lowest prices in a range to understand how expensive a product or brand is. So, by removing high price point lines with low volume, customers will see more minor price points around the store. Brands must merchandise entry price points to help customers identify the lowest prices and improve the perception of their product ranges. 

    Product Range
    Product Range

    How to adjust Price Perception

    Here are three ways for brands to improve price parity:

    • Marketing to influence Price Perception

    An efficient pricing management strategy will focus on competitiveness and establishing the right price perception among your customers. You can influence customers’ price perception by improving the look and feel of your online stores since simpler designs are often reflections of lower prices. Another great way to influence price perception is to offer loyalty and reward programs that also improve brand loyalty and reinforces the vision of an economy store irrespective of the prices of your products.

    • Competitive Analysis

    Brands can understand price differences after a competitive analysis. Customers often search for similar products across brands to find the best deals, and you will be able to understand customer opinion through competitor analysis.

    • Price Management Automation

    A price monitoring platform can help brands to stay on top of promotions and discounts offered by their competitors. A price intelligence software will help brands associate products by similarity criteria and compare the pricing of their products with those of competitors. It offers a detailed view of the market and ensures that brands take care of their bottom line.

    Conclusion

    When a consumer comes across a similar low-priced product or service from a different brand, they may see it as a good deal or might perceive it not worthy of their time or money. What consumers think about your brand’s price is just as important as the actual price of that product. A buyer may sense a company as “upscale” and assume that they have high prices, or they may see a brand as a discount retailer whose prices are too high for its reputation. At times, consumers might also see cheaper alternatives as inferior. It’s not easy for a brand to understand its customers’ perception of price vs. value it offers. Brands need a long-term, dynamic pricing strategy that matches the demands and trends of a global, competitive market. And in order to drive sustainable growth, they need to make smarter pricing and promotion decisions with insights into competitive pricing. 

    Learn how DataWeave can help make sense of your and your competitor’s pricing & promotional strategies and help your brand build the right Price Perception. Sign up for a demo with our team to know more.

  • 11 Reasons why your eCommerce Business is failing

    11 Reasons why your eCommerce Business is failing

    No matter where your eCommerce business sells, there are some fundamentals that brands have to get right to achieve sales targets. Brands need to find the right product/market fit, nail their lead acquisition strategy, and design a qualified sales funnel to turn prospects into leads and eventually returning customers. They will also have to analyze their customer’s buying journey and get insights into competitors’ strategies to understand what works for their business.

    If your eCommerce business is struggling, read this blog to learn about steps you can take to increase sales and keep your business afloat. 

    1. Lack of social proof

    Customers often check for reviews or testimonials before making a purchase. Our decisions are consciously or unconsciously influenced by the opinions, choices, and actions of people around us. Social proof helps brands build customer trust, adds credibility to their business, improves brand presence, and validates customers’ buying decisions. 92% of consumers are more likely to trust user-generated content (UGC) and non-paid recommendations than any other type of advertising. Additionally, brands should also find ways to combat negative reviews since bad reviews can sometimes be extremely damaging. 

    Understanding these reviews or the impact of your brand’s social proof is critical. At DataWeave, we help brands analyze online reviews to understand customer sentiment and adapt to feedback to enhance their experience with your brand. 

    2. Slow site speed

    Site speed of the home page and checkout page on your D2C website can be a roadblock. Slow sections on your site like My Accounts, checkout, and cart are often overlooked when it comes to tracking site speed. Brands should run their checkout process at least once a month to ensure it’s fast, smooth, and bug-free. You can optimize images, strip unused scripts, implement HTTP/2, etc., to improve site speed and performance. 

    3. Poor customer service

    69% of US consumers say customer service is very important when it comes to their loyalty to a brand. Guaranteeing a return customer is important to maintaining customer loyalty. While the focus is on the first purchase for new customers, your brand’s customer service will determine if first-time customers become repeat buyers. Loyal customers are known to spend 67% more on a brand product than new customers, even if they make up only 20% of your audience. 

    Types of customer service
    Types of customer service

    4. Failure to send traffic to popular products

    Be it your own D2C website, or when selling on a marketplace, you should be able to drive traffic to your best-selling products. One of the best ways for sending traffic to popular products on your website is to run paid ad campaigns and reach new audiences with influencer marketing on social media. Brands can also attract customers with organic media such as writing blogs and producing podcasts. 

    If you’re looking at driving traffic to key products on Amazon & other such marketplaces, sponsored ads are the way to go! Sponsored ads help your best-selling products more discoverable & helps shoppers find your brand with ease

    5. Inadequate pricing

    Finding the right pricing strategy for your eCommerce business is crucial for optimizing sales and increasing revenue. The first step is to perform a competitor and historical data analysis to get a general idea of the market and then develop a pricing strategy that is the right fit for your products. Brands also have to ensure that they have dynamic pricing that can adjust according to supply and demand. 

    Our Digital Shelf solution at DataWeave helps brands track pricing for products across different pack sizes & variants across multiple online retailers and marketplaces helping them stay competitive in the market. 

    Optimize the right pricing strategy
    Optimize the right pricing strategy

    6. Not targeting the right audience

    One of the biggest mistakes that eCommerce businesses can make is targeting the wrong audience. It’s crucial for brands to define that target audience and then tailor products and marketing toward them. To increase sales as an eCommerce business, brands have to understand their audience, their interests, and how to appeal to their interest. Start by creating ideal buyer personas that represent your ideal customers. Also, segmenting audiences and targeting various groups based on buyer personas for ad campaigns will lead to better sales and revenue. 

    Targeting the right audience
    Targeting the right audience

    7. Poor product descriptions

    One of the major and common mistakes by eCommerce brands is using irrelevant product descriptions that are not optimized for the product. Customers don’t add products to their cart if they have difficulty finding sufficient information relevant to the product. Brands should write attention-grabbing descriptions optimized for SEO that are informative for the users. Here are some tips to optimize content to drive more eCommerce sales.

    At DataWeave, our AI-Powered solution helps brands optimize content and visuals across product pages to improve discoverability. 

    8. Not having multiple revenue streams

    Due to COVID-19, many businesses have had to modify or temporarily shut down their daily operations. However, finding new revenue streams can be a great way for eCommerce businesses to make up for the lost income and keep the company afloat. The best solution is to diversify your product offerings by offering commonly purchased products in bundles. 

    9. Low-quality visuals

    Businesses fail to hit their sales targets because of low-quality visuals in product descriptions. High-quality and custom images can improve conversion rates from both marketplaces and image-based channels like social media. Social media users are attracted to exciting, high-quality content that conveys a desirable lifestyle. Brands should use high-resolution, attractive pictures of their products. Brands can also utilize UGC and influencers to help build up their content libraries.

    Low-quality visuals
    Low-quality visuals

    10. Wrong Assortment. Poor Availability

    When your target audience lands on your eCommerce store and cannot find what they’re looking for, it leads to a poor shopping experience, but more importantly a lost sale for your brand! While you cannot have endless inventory, it’s essential to optimize your assortment & product availability to decrease the chances of your customer walking away. Assortment & availability optimization begins with analyzing current and historical inventory trends. If done manually, assortment can be a time-consuming task. A healthy assortment can increase retail sales by creating a positive shopping experience for your customers and encouraging them to return to your store again.

    11. Bad eCommerce UX

    Offering a sub-standard user experience is a common reason why eCommerce businesses find it difficult to increase sales. According to a study, the conversions can fall by up to 7% for every one-second delay in page load time. Businesses can use a countdown clock on their landing page and exit pop-ups to improve conversations. Your landing page and product descriptions should provide information that helps your users make a better and more informed decision. 

    Conclusion

    If your eCommerce’s business sales are tanking, improving site speed, customer service, social proof, and product descriptions are some of the levers you can pull to remedy the situation. Brands should also work on improving online reviews & ratings, availability, assortment, visuals, and website UX to improve customer experience. These steps not only increase loyalty but also improve customer retention. 

    Need help tracking online pricing for your eCommerce business? Or decoding customer sentiment from reviews they’ve left for your products? Or do you need insights into your product assortment and availability? Sign up for a demo with our team to know how DataWeave can help!  

  • How VCs and Brand Rollups are using Data for faster Acquisitions

    How VCs and Brand Rollups are using Data for faster Acquisitions

    When it comes to brands – the biggest story of 2021 was the astronomical growth of Brand Roll-ups. For the uninitiated, Brand Roll-ups are companies that acquire multiple digital consumer brands and then scale these brands 100x by leveraging their own operational expertise across eCommerce platforms, Supply Chain, Warehousing, Marketing, and so on.

    Thrasio is the poster boy for the Brand roll-ups and is valued at over 10 Bn USD.

    Brand rollups have raised over $12 billion in 2021 and the trend only seems to be accelerating in 2022. Not only Brand Roll ups, but VCs too have been pouring money into digital brands. In India, 77+ brands have raised more than 2B USD in 2021. In the US this number is estimated to be north of $10 billion.

    Cumulative capital raised by Amazon Aggregators
    Cumulative capital raised by Amazon Aggregators

    Scaling fast doesn’t come easy. It comes with its own set of challenges. So even with ample experience in running and scaling brands, Brand roll-ups are posed with unique challenges.

    Challenge of Scouting the right brand

    There are 1000s of online consumer brands and new ones are launching every day. Every Brand roll-up wants to be the first one to scout a brand – but this is not easy.

    The challenge here is to identify & pick the right brands without having access to any sales or financial data. Every Brand Rollup has a wishlist with regards to the number of SKUs, price points, reviews, and ratings as well – but don’t have tools in place to scout brands with these criteria in mind. And across multiple platforms and categories, the problem gets more complicated.

    This is an ongoing problem since a brand that was not selling well yesterday may start hitting higher sales numbers a week down the line – and that is why Brand scouting has to be a continuous process.

    One way these aggregators have solved this challenge is by offering mouth-watering referral fees for referring a brand. But this is not a sustainable long-term solution.

    Data Comes to the Rescue

    What Brand Roll-ups need is a continuous and automated data first Brand Scouting solution to enable them to scout the right brands.

    • What are all the brands in a category of interest?
    • Which of these brands is within the filters of Number of SKUs, Price Range, etc.?
    • Which brands have shown an exceptional rise in search rankings?
    • Which brands have shown the most increase in the number of ratings and reviews?
    • Which brands have the highest gain in the customer ratings?
    • What are the estimated sales and market share of the brands?

    DataWeave’s Brand Scouting solution solves exactly this.

    DataWeave’s Brand Scouting Solution

    DataWeave’s Brand Scouting Solution is a comprehensive solution to help Brand Rollups and VCs scout for the ideal brand that fits their acquisition profile. We leverage public data collected from multiple eCommerce platforms to get them the desired information on brands they’re looking for.

    For all the focused categories (Typically 30-40) – we collect data of all the SKUs (Typically 15,000-20,000) and aggregate that at a Brand level:

    • Ranking – Usually Brand Rollups are not interested in the Brands which are on the first page. But, they are interested in the brands which might be b/w 500 to 10,000 ranks but are showing an exceptional gain in ranking week on week.
    Brand Discoverability & Ranking on Amazon
    Brand Discoverability & Ranking on Amazon
    • Ratings – It’s important to look at brands that are showing high improvement in ratings or have consistently shown high ratings. The proportion of 5 stars vs. 1 star is an important metric here.
    • Number of Reviews and Ratings We enable you to find brands that have both high ratings as well as a high number of reviews. This is a very good metric to find the brands in a category that are getting exceptional customer love.
    Brand Popularity Tracker
    Brand Popularity Tracker
    • Filters – We enable filtering in terms of – No. of SKUs, Price Range, Rating and Reviews and even can eliminate established brands so that you only see the brands which qualify your criteria. We also enable you to separately analyze brands that are buying sponsored ads in a category, so you have a clear distinction between organic and sponsored growth of these brands.
    • Trends – What is important is not just the static performance on the day of analysis – but a trend analysis over a period of time to find the brands which are growing exceptionally.
    Brand Score Trend, Average Rating trend & No of Reviews Trend
    Brand Score Trend, Average Rating trend & No of Reviews Trend

    … but, wait there’s more.

    We compliment Brand Scouting with three more solutions to provide the right context and further analysis needed to provide comprehensive insights into the category and platforms where you are scouting for brands:

    Category Analytics: When you are looking at a category and the brands in that category, it is often important to understand how dynamic that category is. We can help analyze:

    • If the category is crowded with more brands per product.
    • Does it have space for new brands?
    • What is the number of new brands entering that category?
    • What is the number of new SKUs entering that category?
    Category & Subcategory Evaluation
    Category & Subcategory Evaluation

    We can also help with benchmarking the category – to help understand how the brand that you are scouting is doing when compared to its category peers.

    Rank Group versus Price, Rating & No of Reviews
    Rank Group versus Price, Rating & No of Reviews

    Sales & Share: We can also provide a good directional estimate of the sales and market share of all the SKUs in the category wherein you are scouting for brands. These are estimates powered by our proprietary machine learning algorithms and can help you solidify your hypothesis around a blog or a category.

    Revenue by Price Points
    Revenue by Price Points

    Sentiment Analysis of Reviews: Customer reviews tell more about the qualitative aspects of the SKU and the brand itself. Our algorithms can help understand what features of a brand or a product do customers really care about. We can answer questions such as:

    • Which features are mentioned most commonly?
    • Which features are mentioned positively or negatively?
    • What adjective is used to describe that particular feature?
    Customer Sentiment Analysis
    Customer Sentiment Analysis

    The suite of Brand Scouting and complementary solutions is evolving rapidly as the space is evolving rapidly. We are supporting several VCs and Brand Roll-ups globally to scout for brands.

    The best aspect about DataWeave is our ability to scout brands across 2,000+ eCommerce platforms globally across geographies. We are super stoked to be playing an enabler in the Brand Rollup revolution.

    Beyond Brand Scouting – Digital Shelf Analytics

    The challenge for Brand roll-ups is not over by just scouting and acquiring a brand. The journey is just about starting – the next challenge that the Brand Rollup faces now is to scale up these brands.

    The challenge the Brand Rollup face is unique and very different from a single brand operator or even traditional CPG conglomerates.

    DataWeave’s flexible product philosophy enables Brand Roll-ups to diagnose and measure the performance of multiple brands across multiple platforms in one dashboard.

  • What Historical Pricing Data can tell you & how to use it

    What Historical Pricing Data can tell you & how to use it

    For many brands, pricing strategy boils down to guesswork — shooting in the dark and hoping consumers are willing and happy to pay. However, the ‘throw it at the wall, and see what sticks’ pricing strategy leads to big pricing mistakes. Pinning down an optimal price for products requires a clear picture of ideal customers, understanding each customer segment’s behavior, a solid grasp of your product’s value, and an analysis of competitors. Pricing analytics can help brands track a wide range of pricing metrics with cutting-edge analytical tools and use insights to get ahead of their competition. This analysis uses historical data to understand how previous pricing and promotion activities affect brand, sales, and customer price perception. It often involves identifying opportunities and weaknesses in competitors’ pricing strategies and exploiting them to improve sales and revenue. 

    Pricing analytics helps brands understand how product pricing and promotions affect profitability and the steps they can take to optimize their pricing structures. Brands can leverage their pricing and consumer data to design appropriate pricing models for achieving their sales goals.

    Here is a brief overview of pricing analytics, its benefits, and ways to improve sales with historical pricing analytics.

    What is historical pricing data analytics?

    historical pricing data analytics
    Historical Pricing Data Analytics

    Pricing analytics uses historical pricing and demand data to understand how pricing activities have affected profitability and overall brand. It also helps to optimize a brands’ pricing strategy for maximum revenue. Manual tracking of pricing for brands with numerous product lines, multiple selling points, different customer tiers, and complex product bundles is a huge challenge. Brands from every sector and industry vertical, manufacturing and distribution to retail and eCommerce, can benefit from pricing analytics.

    There are three types of pricing analysis:

    Descriptive

    Descriptive pricing analytics involves analyzing historical data to evaluate how customers have perceived and reacted to pricing fluctuations in the past. It analyzes metrics such as month-on-month sales growth, average revenue per customer, year-on-year pricing changes, or changes to the number of registrations to a particular service over a specific period. 

    Predictive

    Although brands can’t accurately predict how pricing changes will reflect sales, they can use predictive pricing analytics to get insights into the best possible chance of doing so. Predictive pricing analyzes historical data with statistical algorithms and machine learning to predict the price and trends of products in the future. It also helps brands to optimize their prices with future goals.

    Prescriptive

    Prescriptive pricing analytics is the opposite of descriptive analytics. Unlike descriptive analytics that helps brands explore their historical data to understand customer response after an event, prescriptive analytics help brands design better and more informed strategies. With prescriptive analytics, brands can shape their growth strategies to achieve more sustainable results over the long term.

    Benefits of historical pricing data analytics

    Benefits of historical pricing data analytics
    Benefits of Historical Pricing Data Analytics

    Acquire insights into customers price perception

    While analyzing the metrics to understand pricing optimization, brands can also gather valuable insights into their customer’s price perception. Pricing analytics helps brands understand which customer segments are the most (and least) profitable and how each segment responds to specific pricing strategies. With historical pricing data analytics, brands can also intelligently link pricing and promotions by first determining customer price sensitivity then gauging the effectiveness of promotions

    Fully Optimized Pricing

    Historical pricing analytics means eliminating guesswork from deciding the optimal pricing for a given product. By analyzing historical pricing data, brands can discover how their past pricing and promotional decisions impact profitability. Based on this historical data, they can also test various pricing strategies like value-based and dynamic pricing. It also allows brands to learn which customer segments are most likely to respond positively to price change. These insights from pricing analytics will drive more effective (and profitable) pricing decisions.

    Recognize pricing tiers that work the best

    Many brands have gaps in their pricing strategy — underpriced or overpriced tiers, pricing leaks, markup errors, or neglected upsell opportunities. Tiered pricing models are prevalent in subscription-based brands where brands offer tiers to meet the needs of diverse customer segments. With historical pricing analytics, brands can improve their pricing tiers and get insight into the right number of tiers and optimal prices for each. Pricing analytics will comb a brand’s historical data to find tier pricing mistakes to improve sales and revenue.

    Planning Pricing Strategies and Promotions

    Promotional pricing decisions are critical for any brand, as pricing perception is directly linked to consumer demand and profits. Brands have to carefully plan promotions that include variables such as list prices, special offers, advertisements, and discounts while ensuring profit margins. With predictive analytics, brands can determine optimal discount levels, keep a close eye on the competition, and announce promotional offers when customers are likely to purchase. Historical pricing analysis also helps predict revenue and determine optimal locations and platforms for promotional ads.

    Discover profitable channels

    Not all sales channels bring equal revenue to your brand. Historical pricing analysis can help you determine the most effective quality, volume, and revenue channels. Brands must understand which marketing and sales channels bring quality leads that convert to paying customers. It also helps to determine which eCommerce channels are most profitable so you can optimize your budget and identify channels you should be investing in as a part of future customer acquisition strategies. 

    Metrics to track

    Metrics to track
    Metrics to Track

    Here are a few pricing analytics metrics that can help brands to understand customer behavior towards pricing:

    Willingness to Pay (WTP)

    WTP, also known as price sensitivity, is the maximum price your potential customers are willing to pay for your service or product. It is an essential part of pricing strategy since you have no other way of understanding whether your product can yield an augmented product value. Numerous factors are responsible for a customer’s willingness to pay, and it’s not static. Brands must track willingness to pay for all customer segments to ensure that the product is priced competitively and drives maximum profit while staying in line with current market conditions. 

    Feature Value Analysis

    Feature value analysis, also known as relative reference analysis, measures the most important features to customers in relation to other features of a product or service. Analyzing critical features to customer segments will help brands price products based on basic or premium components. It can also help to better bundle your services or products so you can drive the most revenue. 

    Average Revenue per User (ARPU)

    The average revenue per user is the revenue generated from the sum of active users divided by the total number of users in a monthly time frame. Delving deeper into ARPU can help brands compare numbers with rivals and check how all products or customer segments perform. 

    Lifetime Value (LTV)

    Lifetime Value offers a complete picture of a user’s journey and the average revenue that the user will generate throughout their engagement as a customer with your brand. It helps brands determine various economic decisions such as marketing budgets, profitability, forecasting, and resource allocation. 

    Customer Acquisition Cost (CAC)

    A successful and profitable brand needs to balance its customer acquisition cost or CAC. It is about spending the right amount of resources and time to drive new customers without jeopardizing their lifetime value and revenue. Correct calculation of CAC helps brands to quantify their sales funnel and determine the efficiency and profitability of their strategies.

    Conclusion

    Historical pricing analytics is a powerful tool, and it can make a huge difference to a brand’s potential by increasing sales and unlocking incredible profitability in a relatively short time. Historical analysis of pricing and promotions data will help brands get better marketing returns than relying on traditional pricing approaches. 

    Leveraging pricing analytics will prevent brands from blindly reacting to competitor price changes and support solutions for scaling up price transformation efforts. By using historical pricing data, brands can more effectively segment their customers for marketing and promotion strategies. Properly utilizing predictive analytics and past sales data can help cut costs and keep profit margins high by adjusting production and prices according to market trends.
    Need help tracking your competitor prices? Or want historic pricing insights for your own brand? Or need to track the efficacy of your online promotions?
    Sign up for a demo
    with our team to know how DataWeave can help!

  • Best Practices to avoid MAP Violations

    Best Practices to avoid MAP Violations

    Competition is a fundamental and healthy part of commerce that protects customers by keeping prices low and the quality of services (and choice of goods) high.

     Healthy competition drives prices down, but it can harm brands and their reputation without a pricing policy. The manufacturer or brand designs MAP or Minimum Advertised Pricing policies to stipulate retailers’ lowest price point to advertise the product. It is an agreement between distributors and manufacturers about the minimum price that retailers and resellers can advertise the product for sale. 

    Most legitimate brands have a MAP policy, especially brands that rely heavily on brand identity. It becomes critical that they maintain price parity across retailers. When a retailer violates MAP policies, brands can penalize them under the agreed-upon terms or terminate contracts. 

    In this blog, you will learn about MAP policy, its benefits, and tips on tackling MAP violations. 

    1. What is a MAP policy?

     MAP policy
    MAP Violations

    MAP stands for Minimum Advertised Price, and brands create MAP policies to ensure that retailers don’t advertise their products below the specified price. However, it only controls advertised prices, ensuring the retailers don’t display a lower price in online listings or advertisements. Since it doesn’t cover the checkout price, retailers can sell products at a lower price through promotional offers like discounts and cashback during checkout. 

    MAP policies ensure a price war between eCommerce platforms does not devalue products and that an even playing field is set among retailers that allow everyone to drive margins. Brands have a legal right to withdraw products if a retailer advertises products below the minimum advertised price. Brands can also restrict future sales or refuse to replenish products after the current stock has sold out if an eCommerce platform, reseller, or distributor violates MAP policies. 

    In the U.S., MAP policies fall under federal antitrust law since they restrict advertisement pricing rather than the last sales price. However, in the UK and the EU, violation of minimum advertised pricing is an infringement of current competition laws.

    2. Why Does Having a MAP Policy Matter?

    Having a MAP policy protects both brands and retailers while ensuring consumers get the best-priced items. Following are the benefits of having a MAP policy:

    a. Prevent margin erosion

    Although online retailers are willing to take a margin cut to attract traffic, selling products below MAP can significantly hurt a brand’s bottom line. Setting a minimum advertised price benefits both parties. It allows shoppers to purchase products at the best-valued price & also creates a balanced economy and prevents hyper-competition of products between retailers. However, manufacturers must set a realistic pricing policy that matches current market demand, ensuring eCommerce platforms implement MAP while taking care of the margins. 

    b. Retain brand identity

    pricing policy
    Brand Protection

    Price is one of the essential indicators consumers use to determine the authenticity and value of a product. Constant price fluctuations can negatively impact a brand’s reputation. Brands need to safeguard their pricing to create a consistent price perception. Price changes often make the buying decision complex since consumers no longer have a clear reference of prices. It also shifts purchasers’ attention from the brand and product features to its price. With price fluctuations, brands that were used to be differentiated for their features can be seen as commodities.

    Low prices & MAP violations on an online platform can even be a sign of counterfeit products or unauthorized sellers. However, customers might hold the brand responsible if they purchase counterfeit products from a retailer at lower prices. A negative product experience with a retailer will also reflect the brand’s reputation. An effective MAP policy that enforces consistent pricing will ensure that customers hunting for the best deals will stick with the most legitimate retailers.

    Read how DataWeave helped Classic Accessories, a leading manufacturer of high-quality accessories detect counterfeits and identify unauthorized sellers.

    c. Ensure price parity across retailers

    Comparing prices has become an essential and common milestone in every consumer’s purchasing journey. It’s imperative that a brand ensures price parity across platforms and stores because substantial pricing variations on different platforms can make customers suspicious of a brand. Consistent pricing across eCommerce platforms ensures brands maintain their identity. MAP policies also allow retailers to maintain profit margins while avoiding price wars.

    d. Combat revenue loss from illegitimate sales

    While most authorized sellers or distributors comply with pricing policies, unauthorized sellers or grey market sellers have no obligation to follow a brand’s MAP pricing infrastructure. Brands can reduce risk with an authorized seller badge on retailer websites. This will help customers to verify authorized retailers and resellers of your products & help safeguard your brand equity online

    3. Tips on Implementing MAP policy and Tackling violations

    Enforcing and tackling MAPs comes down to two things: monitoring the market for infringements and then acting on those violations. Here are a few tips for tackling MAP violations:

     price parity
    Implementation of MAP Policy & Tracking Violations

    a. Communicate actively with retailers

    To maintain a positive relationship with retailers and avoid confusion, brands should create proper communication strategies and channels to accompany the launch of the MAP policy. The policy should be easy to understand, but legal advisors are necessary to understand the jargon of the document. Brands can use checklists, videos, and well-briefed brand reps to communicate their policy clearly with retailers.

    b. Reward retailers for compliance

    Retailers who follow MAP guidelines can lose out to platforms that do not follow these pricing guidelines. Non-MAP following platforms undercut the price of products to drive sales and secure higher traffic. In such instances, brands can incentivize MAP following retailers to encourage them to comply with MAP guidelines while not affecting the competitive edge. It can be in the form of laxity of rules during promotion seasons like New Year, Christmas, and Black Friday sales. The laxity of rules for promotional seasons should be used as an exception to the general rule, and outlined in the guidelines.  

    c. Implement an AI-driven MAP monitoring

    When product distribution is spread across the globe through a network of resellers and retailers, keeping a close watch on all platforms for multiple products can become difficult. With the expansion of online marketplaces, manually tracking the pricing of numerous products on multiple platforms is time-consuming and unsustainable. An automated AI-driven monitoring platform can track the pricing of all products sold across hundreds of online platforms and identify violations around the clock. Such platforms can alert brands of violations, price inconsistencies, or suspicious activities in real-time. 

    d. Send cease and desist to MAP violators and unauthorized dealers

    Brands must enforce a MAP policy to ensure price parity among retailers and resellers. Brands must systematically monitor prices across retailers, social media, marketplaces, and price comparison websites. Whenever brands encounter a MAP violation, they should take action by sending a cease and desist letter to unauthorized sellers. For legitimate sellers, brands can notify them and outline the steps that will be taken if they don’t comply. Brands must be consistent in enforcing MAP policy violations, signaling retailers and unauthorized sellers that there will be repercussions for MAP violations. 

    Market Demand
    MAP Policy

    4. Conclusion

    The trend towards online shopping helps businesses to cut overheads, allowing their products to be sold at a significantly reduced price. Although price appears to be the most effective consumer attraction strategy, significantly lowering product prices can devalue products and hurt brand reputation in the long term. However, including and enforcing MAP policies helps brands to manage their reputation and allows retailers to manage their margins. 

    Want to see first-hand how DataWeave can help brands track MAP Violations, Counterfeit products, and identify unauthorized sellers? Sign up for a demo with our Digital Shelf experts to know more.

  • Importance of Image Recognition in the Retail Industry

    Importance of Image Recognition in the Retail Industry

    When it comes to classifying and analyzing images, humans can easily recognize distinct features of objects and associate them with individual definitions. However, visual recognition is a highly complex task for machines because it involves identifying multiple objects and finding object relationships. Image recognition has been a long-standing research problem in the computer vision field. But, the recent development in AI has improved the process of object detection, image identification, and image classification. The image recognition market is assumed to rise globally to a market size of $42.2 billion by 2022. Various industries are adopting image recognition technology to improve augmented reality applications, optimize medical imagery, boost driverless car technology, predict consumer behavior, and much more. 

    Although image recognition is a relatively new aspect of analysis, it is also making its way into eCommerce. Image recognition is helping retailers to expand consumer reach, offer insights into trends, and improve customers’ online shopping experience for the eCommerce industry. The Global Image Recognition in Retail Market is estimated to be USD 1.8 Bn in 2021 and is expected to reach USD 4.5 Bn by 2026, growing at a CAGR of 20%.

    Image Recognition
    Global Image Recognition in Retail Market

    In this blog, you’ll learn about image recognition technology and its importance in the retail industry. 

    What is Image Recognition?

    Image recognition, a subcategory of computer vision, is a technology that can identify objects, entities, or attributes in digital images or videos. However, computer vision is a broader term, including methods for gathering, processing, and analyzing data from the real world. Image recognition can be performed at varying degrees of accuracy, depending on the type of information required.

    Image recognition can perform the following tasks:

    Object Detection, Semantic Segmentation &  Instance Segmentation
    Object Detection, Semantic Segmentation & Instance Segmentation
    • Classification: It identifies the “class,” i.e., the category to which an image belongs. A picture can have only one class.
    • Tagging: It’s a classification task but involves a higher degree of accuracy. Tagging can recognize several concepts or objects within an image, and there can be more than one tag assigned to a particular image.
    • Detection and localization: This step helps locate object(s) in an image. Once the system locates the object in question, localization helps to place a bounding box around it. 
    • Segmentation: This is also a detection task but involves a higher degree of precision. Segmentation locates element(s) to the nearest pixel in an image. 
    • Instance segmentation: It helps differentiate multiple objects belonging to the same class. 

    Image Recognition in eCommerce and how it works

    Nowadays, increasing competition and customer expectations are forcing online retailers to constantly monitor market dynamics wrt their pricing, promotion & product assortment in order to stay competitive. To get these insights, retailers need to match and compare their products against their competitors to see where the gaps are. That’s where product matching comes in. 

    Product matching refers to finding the same or similar products against a target universe of products from across the web, across multiple competing retailers. Product matching uses AI-based image recognition to determine product attributes, find patterns, and detect text, product price, shipping information, and so on. 

    Here’s how DataWeave’s AI-powered analytics platform uses image recognition & aggregates insights & data for retailers from across the web to provide a comprehensive view of the online competitive environment.

    Image recognition use-cases in the retail industry

    a. Attribute tagging

    Attribute Tagging
    Attribute Tagging

    Getting shoppers to your eCommerce platform is one thing and getting them to complete a purchase is a steeper hill to climb. If your platform can’t provide search results that match with customers’ requirements, they’ll get lost, grow frustrated, and drop off. Attribute tagging with image recognition allows eCommerce stores to automatically generate attributes for all products so customers can quickly find products they are looking for. 

    Tags allow users to filter products based on the categories they want to explore. Product tags include everything the customer might specifically search for — color, type, size, brand, use, design, fabric, discount, etc. For example, a dress could have tags like red, evening, midi, summer, long-sleeve, silk, summer sale, etc. When a user looks for midi dresses or long-sleeve dresses, products with these tags will show up. 

    b. Search by image

    Visual Search
    Visual Search

    Visual Search allows users to look for similar products using a reference image from their camera roll or downloaded from the internet. The visual search feature also enables eCommerce businesses to implement image-based search into their software applications. It maximizes the searchable potential of their visual data. 

    Meanwhile, Gartner predicts a 30% increase in digital commerce revenue by 2021 for companies who start supporting visual and voice search on their websites and apps. The benefits of visual search include more personalized, easy product recommendations and enhanced product discovery.

    c. Fashion trend analysis

    similarity matching
    Similarity Matching

    Tapping into trending product categories is a goldmine for any eCommerce business. Having insights into trending categories and products means less competition on search engines, fewer ads, and intelligent pricing. All of which can boost any retailer’s margins. Image recognition technology provides information about colors, styling techniques, fabric textures, prints, and more to spark consumer demand. It works by scanning social media images to pinpoint trending attributes and predict fashion trends. For instance, while scanning images, technology understands that it’s seeing a photo of a color-blocked sweatshirt because it recognizes the product has a hooded neck, full sleeves, blocks of different colors, and even the type of fabric. This technology can analyze millions of images, helping retailers analyze the volume of color-blocked sweatshirts. 

    We do this seamlessly at DataWeave. Our similarity matching solution helps retailers gather insights into attributes for products similar to the ones they’re carrying on their site. Similarity matching helps retailers gain visibility into their entire competitive landscape to keep their e-commerce strategy responsive to price & product assortment shifts among consumers and rivals

    d. Augmented reality

    According to Statista, the AR market is valued at $9.5 billion, with around 810 million active mobile users. Since shoppers want the full sensory product experience before shopping online, augmented reality (AR) can help them understand what they’re buying and how the product will work for them. There are AR applications for trying makeup, clothing, accessories, and even eyeglasses. IKEA was one of the pioneers in using AR for eCommerce retail. In 2017, IKEA launched the Place app, allowing shoppers to see how thousands of items will look in their homes, with 98% accuracy. 

    Image recognition helps AR applications anchor virtual content with the real world. For instance, Sephora has a Virtual Artist that allows users to try different makeup looks and even take pictures of an outfit they’re planning to wear to match the shade. Users can even check out full-face looks and learn how to do their makeup with virtual tutorials. 

    e. Counterfeit Detection

    Counterfeit Detection
    Counterfeit Detection

    Another application of image recognition that has proven to be very successful is counterfeit product detection. It has become increasingly difficult for brands and retailers to find and eliminate fake items on eCommerce sites. U.S. Customs seized over 13,500 counterfeit goods worth $30 Million in November 2021, indicating how brands and online marketplaces have struggled in the past to find an effective solution. 

    Essentially, image recognition technology allows eCommerce sites to detect products with fake logos and designs attempting to sell as legitimate brands by capturing discrepancies in images and content. The system flags and delists the products and sellers when a fake is detected.

    Here’s how DataWeave helped Classic Accessories, a leading manufacturer of high-quality covers, furnishings, and accessories automate their counterfeit detection process using our super Image Recognition capabilities. 

    f. User-generated content analysis

    Visual content plays a vital role in eCommerce sites, especially when it comes to product photos and videos. Today, branded visual content isn’t as effective as it’s one-dimensional. As a matter of fact, 93% of marketers agree that customers trust user-generated content more than content produced by brands. However, user-generated content that features product images or videos is way more exciting, realistic, and creative. It gives customers an appealing view of products being used in real life. 

    The most common form of UGC, i.e., reviews and ratings, have been the key for eCommerce brands as they are quantitative and qualitative metrics about a product/service quality, worth, value, reliability, etc. With image recognition, retailers can access insights into strengths and gaps in all product offerings by understanding what consumers are saying about them. 

    Here’s how DataWeave can help retailers and brands analyze consumer reviews & help them adapt to customer needs.

    Conclusion

    Because of its massive influence, image recognition technology is becoming widely adopted by eCommerce companies. It benefits both retailers and customers. Image recognition based on deep learning can provide retailers with helpful capacities like customer analytics, counterfeit detection, personalized searches, and more. Retailers can also use the data gathered from image recognition eCommerce technology to design effective marketing campaigns and improve their ROI.

    With super sharp image recognition capabilities, DataWeave offers 90% accuracy in matching eCommerce products, allowing us to provide comprehensive and precise insights into pricing and assortments. Sign up for a demo with our team to know more.

  • Top 7 AI tools for your eCommerce business

    Top 7 AI tools for your eCommerce business

    The 2020 global health crisis sped up the adoption of omnichannel shopping and fulfillment. Consumers spent $791.70 billion online with U.S. merchants in 2020, a 32.4% rise compared to 2019. To keep up with this digital shift, offline businesses have substantially moved investments to online infrastructures for everything from e-commerce platforms, product recommendations, inventory management, and communications. AI tools for eCommerce have played a major role in helping businesses in the digital shift. 

    However, the benefits of setting up e-commerce stores are potentially outweighed by the increased costs. As markets transition to online retailers, they must learn to efficiently collect, secure, and analyze data coming in from multiple sources. Strategically approaching the data problem with artificial intelligence (AI) can help better serve customers, gain a competitive advantage, and drive loyalty.

    In this blog, you will learn about seven data and AI tools for eCommerce businesses:

    Seven data and AI tools for eCommerce businesses
    Seven Data and AI tools for eCommerce businesses

    1. Data Warehouse

    Data is the one advantage that eCommerce merchants and marketers have over brick and mortar retailers. When buyers are from the internet, eCommerce retailers can collect data and measure almost every aspect of their interactions. However, that advantage is worthless unless there is a system to make sense of the data they collect. Companies assume that they have a sound system in place. But, what they have is a network of silos. In such a system, data sticks to different platforms like Google Analytics, Shopify, or Klaviyo and can’t move to deliver valuable insights. Funneling all your data into a single location for your eCommerce stores is the right way to go. Data warehouses centralize and merge a plethora of data from various sources, helping organizations to derive valuable business insights and improve decision-making. 

    Data Warehouses support real-time analytics and ML operations quickly & are designed to enable and support business intelligence (BI) activities like performing queries and analysis on a colossal amount of data. Data could range from customer-related data, product or pricing data, or even competitor data. 

    However, the time needed to gather, clean, and upload the data to the warehouse is a time-consuming process. Here’s where DataWeave’s AI-Powered Data Aggregation & Analysis Platform can help! Get critical insights on your competitor’s pricing, assortment, and historical sale trends with a real-time dashboard. Build a winning eCommerce strategy with market intelligence without the need to store your data. 

    2. Data Lake

    Data Lake

    A data lake is a centralized repository that can store structured and unstructured data at any scale. Companies don’t have to provide a schema to the data before storing it, but they still can run different analytics and ML-related operations. However, it takes more time to refine the raw data and then analyze or create ML models for predictions. 

    An Aberdeen survey saw businesses implementing a Data Lake outperforming similar companies by 9% in organic revenue growth. The organizations that implemented Data Lake could perform various analytics over additional data from social media, click-streams, websites, etc. A Data Lake allows for the democratization of data and the versatility of storing multi-structured data from diverse sources, improving insights and business growth. 

    eCommerce businesses can collect competitors’ data in data lakes like their popular products, categories, landing pages, and ads. Analyzing competitors’ data helps retailers price their products correctly, helps with product matching, historical trend analysis, and much more. However, data lakes can also be used to store consumer data such as who they are, what they purchase, how much they spend on average, and how they interact with a company. Successful retailers leverage both competitor and consumer data to understand their consumers better, what brands to carry, how to price each product, and what categories to expand or contract. Retailers also store identity data such as a person’s name, contact information, gender, email address, and social media profiles. Other types of data stored are website visits, purchase patterns, email opens, usage rates, and behavioral data. 

    The major challenge with a data lake architecture is that it stores raw data with no oversight of the contents. Without elements like a defined mechanism to catalog and secure data, data cannot be found, or trusted resulting in a “data swamp.” Consequently, companies need teams of data engineers to clean data for data scientists or analysts to generate insights. This not only increases the turnaround time of gaining valuable information but also increases operational costs.

    However, you can rely on platforms like DataWeave that stores competitor pricing & assortment information at a centralized location. You can leverage intelligently designed dashboards to get real-time insights into the collected data and make data-driven decisions without the need for storing, cleaning, and transforming the data.

    3. Data Ingestion & ETL

    To churn out better insights, businesses need access to all data sources. An incomplete picture of data can cause spurious analytic conclusions, misleading reports and inhibit decision-making. As a result, to correlate data from multiple sources, data must be in a centralized location—a data warehouse or a data lake. However, extracting and storing information into these systems require data engineers who can implement techniques like data ingestion and ETL.

    While data ingestion focuses on getting data into data lakes, ETL focuses on transforming data into well-defined rigid structures optimized and storing it into a data warehouse for better analytics workflows. Both processes allow for the transportation of data from various sources to a storage medium that an organization can access, use, and analyze. The destination can be a data warehouse in the case of ETL and a data lake in case of data ingestion. Sources can be almost anything from in-house apps, websites, SaaS data, databases, spreadsheets, or anywhere on the internet.

    Data ingestion & ETL are the backbones of any analytics/AI architecture since these processes provide consistent and convenient data, respectively. 

    4. Programming languages

    Programming languages

    Programming languages are tools used by programmers to write instructions for computers to follow since they “think” in binary—strings of 1s and 0s. It serves as a bridge that allows humans to translate instructions into a language that computers can understand. Some common and highly used programming languages for building AI models are Python and R.  

    While Python is the most widely used language for training and testing models, R is mostly embraced for visualizations and statistical analysis. However, to productize the ML models, you would require Java programming language so that models can be integrated with your websites to provide recommendations.

    5. Libraries/AI frameworks

    An AI framework is a structure that acts as a starting point for companies or developers to add higher-level functionality and build advanced AI software. A framework serves as a foundation, ensuring that developers aren’t starting entirely from scratch.

    Using AI frameworks like TensorFlow, Theano, PyTorch, and more saves time and reduces the risk of errors while building complex deep learning models. Libraries and AI frameworks also assist in building a more secure and clean code. They future aid developers in simpler testing and debugging.

    Various open-source frameworks in the market also come with pre-trained models for specific use cases. Organizations can leverage off-the-shelf models and tweak with existing data to enhance the accuracy of the predictions.

    6. IDE & Notebooks tools

    IDE or Integrated Development Environment is a coding tool that allows developers to write and test their code more efficiently. However, notebooks are one of the most popular AI tools for organizations to execute analysis and other machine learning tasks. It offers more flexibility over IDEs in terms of exploratory analysis.

    All the features, including auto-complete, that IDEs or notebooks offer are beneficial for development as they make coding more comfortable. IDEs/Notebooks increase developers’ productivity by combining common software activities into a single application: building executables, editing code, and debugging.

    7. Analytics tools

    Competitive Pricing

    Data Analysis transforms raw data into valuable statistics, insights, and explanations to help companies make data-driven business decisions. Data analytics tools like PowerBI and Tableau have become the cornerstone of modern business for quickly analyzing structured and semi-structured data. 

    However, these platforms aren’t optimized specifically for the eCommerce industry. Consequently, you should embrace analytical tools particularly designed for eCommerce companies to make better decisions about product assortment, pricing, and promotions. With data analytics, companies can gain insights into the most popular and discoverable brands on their own and competitors’ platforms. Paired with attribute matching, competitive intelligence gives a deeper understanding of the latest trends and why certain products are popular with your customers. Some more meaningful metrics that retailers can track are discount gap, price gap, catalog strength, and product type gaps. 

    Competitive pricing is another benefit of data analytics with which retailers can identify gaps and keep up with actionable pricing insights. Retailers get to maximize profits and respond to demand by cashing in on insights into rivals’ pricing. With the right analytics tools, they can also track changes in pricing across crucial metrics such as matched products, recent price changes, highest price positions, stock status, and much more. 

    Analytics tools can also help eCommerce companies to capture information about competitors’ promotional banners through AI-powered image analysis. It can provide insights into how and where to spend promotional expenditure. 

    Conclusion

    This listicle discusses some of the AI and data tools commonly used by the eCommerce industry. Data analytics has become a popular method for retailers to understand their customers and boost productivity. Data analytics help companies improve customer experience, improve customer loyalty, generate insights, and advise on data-driven actions. Business intelligence tools can help companies monitor key performance indicators (KPIs), perform proper data analyses, and generate accurate reports. 

    Want to learn how DataWeave can help make sense of your and your competitor’s pricing, promotional, and assortment data? Sign up for a demo with our team to know more.

  • How Brands Can Outperform Rivals With Next-Gen Digital Shelf Analytics

    How Brands Can Outperform Rivals With Next-Gen Digital Shelf Analytics

    As eCommerce grows in complexity, brands need new ways to grow sales and market share. Right now, brands face urgent market pressures like out-of-stocks, an influx of new competition and rising inflation, all of which erode profitability. As online marketplaces mature, more brands need to make daily changes to their digital marketing strategies in response to these market pressures, shifts in demand, and competitive trends.

    eMarketer forecasts 2021 U.S. eCommerce will rise nearly 18% year-over-year (vs. 6.3% for brick-and-mortar), led by apparel and accessories, furniture, food and beverage, and health and personal care. The eCommerce industry is also undergoing fundamental changes with newer entities emerging and traditional business models evolving to adapt to the changed environment. For example, sales for delivery intermediaries such as Doordash, Instacart, Shipt, and Uber have gone from $8.8 billion in 2019 to an estimated $35.3 billion by the end of 2021. Similarly, many brands have established or are building out a Direct to Consumer (D2C) model so they can fully own and control their customer’s experiences.

    In response, DataWeave has launched the next generation of our Digital Shelf Analytics suite to help brands across retail categories directly address today’s costly market risks to drive eCommerce growth and gain a competitive advantage.

    Our new enhancements help brands improve online search rank visibility and quantify the impact of digital investments – especially in time for the busy holiday season.”  
    ~ Karthik Bettadapura, CEO and co-founder, DataWeave

    The latest product enhancements provide brands access to tailored dashboard views that track KPI achievements and trigger actionable alerts to improve online search rank visibility, protect product availability and optimize share of search 24/7. Dataweave’s Digital Shelf Analytics platform works seamlessly across all forms of eCommerce platforms and models – marketplaces, D2C websites and delivery intermediaries.

    Dashboard for Multiple Functions

    While all brands share a common objective of increasing sales and market share, their internal teams are often challenged to communicate and collaborate, given differing needs for competitive and performance data across varying job functions. As a result, teams face pressure to quickly grasp market trends and identify what’s holding their brands back.

    In response, DataWeave now offers executive-level and customized scorecard views, tailored to each user’s job function, with the ability to measure and assess marketplace changes across a growing list of online retail channels for metrics that matter most to each user. This enhancement enables data democratization and internal alignment to support goal achievement, such as boosting share of category and content effectiveness. The KPIs show aggregated trends, plus granular reasons that help to explain why and where brands can improve.

    Brands gain versatile insights serving users from executives to analysts and brand and customer managers.

    Prioritized, Actionable Insights

    As brands digitize more of their eCommerce and digital marketing processes, they accumulate an abundance of data to analyze to uncover actionable insights. This deluge of data makes it a challenge for brands to know exactly where to begin, create a strategy and determine the right KPIs to set to measure goal accomplishment.

    DataWeave’s Digital Shelf Analytics tool enables brands to effectively build a competitive online growth strategy. To boost online discoverability (Share of Search), brands can define their own product taxonomies across billions of data points aggregated across thousands of retailer websites. They can also create customized KPIs that track progress toward goal accomplishment, with the added capability of seeing recommended courses of action to take via email alerts when brands need to adjust their eCommerce plans for agility.

    “Brands need an integrated view of how to improve their discoverability
    and share of search by considering all touchpoints in the digital commerce ecosystem.”

    ~ Karthik Bettadapura, CEO and co-founder, DataWeave

    Of vital importance, amid today’s global supply chain challenges, brands gain detailed analysis on product inventory and availability, as well as specific insights and alerts that prompt them to solve out-of-stocks faster, which Deloitte reports is a growing concern of consumers (75% are worried about out-of-stocks) this holiday season.

    User and system generated alerts provide clarity to actionable steps to improving eCommerce effectiveness.
    You also have visibility to store-level product availability, and are alerted to recurring out-of-stock experiences.

    Scalable Insights – From Bird’s Eye to Granular Views

    DataWeave’s Digital Shelf Analytics allows brands to achieve data accuracy at scale, including reliable insights from a top-down and bottom-up perspective. For example, you can see a granular view of one SKUs product content alongside availability, or you can monitor a group of SKUs, say your best selling ones, at a higher level view with the ability to drill down into more detail.

    Brands can access flexible insights, ranging from strategic overviews to finer details explaining performance results.

    Many brands struggle with an inability to scale from a hyper-local eCommerce strategy to a global strategy. Most tools available on the market solve for one or the other, addressing opportunities at either a store-level basis or top-down basis – but not both.

    According to research by Boston Consulting Group and Google, advanced analytics and AI can drive more than 10% of sales growth for consumer packaged goods (CPG) companies, of which 5% comes directly from marketing. With DataWeave’s advanced analytics, AI and scalable insights, brands can set and follow global strategies while executing changes at a hyper-local level, using root-cause analysis to drill deeper into problems to find out why they are occurring.

    As more brands embrace eCommerce and many retailers localize their online assortment strategies, the need for analytical flexibility and granular visibility to insights becomes increasingly important. Google reports that search terms “near me” and “where to buy” have increased by more than 200% among mobile users in the last few years, as consumers seek to buy online locally.

    e-Retailers are now fine-tuning merchandising and promotional strategies at a hyper-local level based on differences seen in consumer’s localized search preferences, and DataWeave’s Digital Shelf Analytics solution provides brands visibility to retailer execution changes in near real-time.

    Competitive Benchmarking

    Brand leaders cannot make sound decisions without considering external factors in the competitive landscape, including rival brands’ pricing, promotion, content, availability, ratings and reviews, and retailer assortment. Dataweave’s Digital Shelf Analytics solution allows you to monitor share of search, search rankings and compare content (assessing attributes like number of images, presence of video, image resolution, etc.) across all competitors, which helps brands make more informed marketing decisions.

    Brands are also provided visibility into competitive insights at a granular level, allowing them to make actionable changes to their strategies to stay ahead of competitors’ moves. A new module called ‘Sales and Share’ now enables brands to benchmark sales performance alongside rivals’ and measure market share changes over time to evaluate and improve competitive positioning.

    Monitor competitive activity, spot emerging threats and immediately see how your performance compares to all rivals’, targeting ways to outmaneuver the competition.

    Sales & Market Share Estimates Correlated with Digital Shelf KPIs

    In a brick-and-mortar world, brands often use point of sale (POS) based measurement solutions from third party providers, such as Nielsen, to estimate market share. In the digital world, it is extremely difficult to get such estimates given the number of ways online orders are fulfilled by retailers and obtained by consumers. Dataweave’s Digital Shelf Analytics solution now provides sales and market share estimates via customer defined taxonomy, for large retailers like Amazon. Competitive sales and market share estimates can also be obtained at a SKU level so brands can easily benchmark their performance results.

    Additionally, sales and market share data can also be correlated with digital shelf KPIs. This gives an easy way for brands to check the effect of changes made to attributes, such as content and/or product availability, and how the changes impact sales and market share. Similarly, brands can see how modified search efforts, both organic and sponsored, correspond to changes in sales and market share estimates.

    Take Your Digital Shelf Growth to the Next Level

    The importance of accessing flexible, actionable insights and responding in real-time is growing exponentially as online is poised to account for an increasing proportion of brands’ total sales. With 24/7 digital shelf accessibility among consumers comes 24/7 visibility and the responsibility for brands to address sales and digital marketing opportunities in real-time to attract and serve online shoppers around the clock.

    Brands are turning to data analytics to address these new business opportunities, enhance customer satisfaction and loyalty, drive growth and gain a competitive advantage. Companies that adopt data-driven marketing strategies are six times more likely to be profitable year-over-year, and DataWeave is here to help your organization adopt these practices. To capitalize on the global online shopping boom, brands must invest in a digital shelf analytics solution now to effectively build their growth strategies and track measurable KPIs.

    DataWeave’s next-gen Digital Shelf Analytics enhancements now further a brand’s ability to monitor, analyze, and determine systems that enable faster and smarter decision-making and sales performance optimization. The results delight consumers by helping them find products they’re searching for, which boosts brand trust.

    Connect with us to learn how we can scale with your brand’s analytical needs. No project or region is too big or small, and we can start where you want and scale up to help you stay agile and competitive.

  • Top 10 Retail Analytics that You Must Know

    Top 10 Retail Analytics that You Must Know

    Customers expect personalization. Unless they have a seamless experience on your online channels, they’ll leave for a different retailer. Retail analytics can solve these problems for merchants looking to increase customer satisfaction and sales. It provides insights into inventory, sales, customers, and other essential aspects crucial for decision-making. Retail analytics also encompasses several granular fields to create a broad picture of a retail business’s health and sales, along with improvement areas.

    Big data analytics in the retail market
    Big data analytics in the retail market

    Big data analytics in the retail market is expected to reach USD 13.26 billion by the end of 2026, registering a CAGR of 21.20% during the forecast period (2021-2026). The growth of analytics in retail depicts how it can help companies run businesses more efficiently, make data-backed choices, and deliver improved customer service.

    In this blog, we’ll discuss the top 10 analytics that retailers are using to gain a competitive advantage in accurately evaluating business & market performance.

    Top 10 of Retail Analytics You Must Know
    Top 10 of Retail Analytics You Must Know

    1. Assortment

    Assortment planning allows retailers to choose the right breadth (product categories) and depth (product variation within each category) for their retail or online stores. Assortment management has grown beyond simple performance metrics like total sales or rotation numbers. Instead, retail analytics offers a comprehensive analysis of product merchandise and an estimated number of units at the push of a button. Retailers that effectively apply assortment analytics can enjoy increased gross margins and prevent significant losses from overstocks sold at discounted prices or out-of-stock inventory leading their customers to buy from competitors. 

    It also helps retailers gain insights into the trendy and discoverable brands and products on all e-commerce websites across the globe. They can boost sales by making sure they have an in-demand product assortment. They can also track pricing information and attributes common across popular products to drive their pricing and promotion strategies.

    2. Inventory Management

    An inadequately maintained inventory is every retailer’s worst nightmare. It represents a poor indicator of inadequate demand for a product and leads to a loss in sales. Data can help companies answer issues like what to store and what to discard. It’s beneficial to discard or increase offers on products that are not generating sales and keep replenished stocks of popular items. 

    Worldwide Inventory Distribution

    In 2020, the estimated value for out-of-stock items ($1.14 trillion) was double that of overstock items ($626 billion). A similar trend was especially prominent in grocery stores, where out-of-stock items were worth five times more than overstock items.

    Unavailability of high-selling products can lead to reduced sales, ultimately generating incorrect data for future forecasting and producing skewed demand and supply insights. Retailers can now use analytics to identify which products are in demand, which are moving slowly, and which ones contribute to dead stock. They can know in real-time if a high-demand product is unavailable at a specific location and take action to increase the stock. Retailers can use this historical data to predict what to stock, at what place, time, and cost to maintain and optimize revenue. It helps satisfy consumer needs, prevents loss of sales, reduces inventory cost, and streamlines the complete supply chain.

    3. Competitive Intelligence

    Market intelligence & Competitive Insights
    Market intelligence & Competitive Insights

    The ability to accurately predict trends after the global pandemic and with an unknown economic future is becoming the cornerstone for successful retailers. Smart retailers know how important it is to Pandemic-Proof their retail strategy with Market Intelligence & Competitive Insights 

    With 90% of Fortune 500 companies using competitive intelligence, it’s an essential tool to gain an advantage over industry competitors. Competitive Intelligence allows you to gather and analyze information about your competitors and understand the market–providing valuable insights that you can apply to your own business. A more strategic competitor analysis will explain brand affinities and provide insights on what to keep in stock and when to start promotions. Customer movement data will also give you access to where your customers are shopping.

    4. Fraud Detection

    Fraud Detection
    Fraud Detection

    Retailers have been in a constant struggle with fraud detection and prevention since time immemorial. Fraudulent products lead to substantial financial losses and damage the reputation of both brands and retailers. Every $1 of fraud now costs U.S. retail and eCommerce merchants $3.60, a 15% growth since the pre-Covid study in 2019, which was $3.13. Retail Analytics acts as a guardian against fraudsters by constantly monitoring, identifying, and flagging fraud products and sellers. 

    5. Campaign Management

    Some of the challenges of the retail industry are that it’s seasonal, promotion-based, highly competitive, and fast-moving. In today’s competitive marketplace, consumers compare prices and expect personalized shopping experiences. Campaign management allows marketing teams to plan, track, and analyze marketing strategies for promoting products and attracting audiences. Retail analytics can help businesses predict consumer behavior, improve decision-making across the company, and determine the ROI of their marketing efforts. 

    According to Invesp, 64% of marketing executives “strongly agree” that data-driven marketing is crucial in the economy. Retail analytics can help businesses analyze their data to learn about their customers with target precision. With predictive analysis, retailers can design campaigns that encourage consumers to interact with the brand, move down the sales funnel, and ultimately convert.

    6. Behavioral Analytics

    Retail firms often look to improve customer conversion rates, personalize marketing campaigns to increase revenue, predict and avoid customer churn, and lower customer acquisition costs. Data-driven insights on customer shopping behaviors can help companies tackle these challenges. However, several interaction points like social media, mobile, e-commerce sites, stores, and more, cause a substantial increase in the complexity and diversity of data to accumulate and analyze. 

    Insider Intelligence forecasts that m-eCommerce volume will rise at 25.5% (CAGR) until 2024, hitting $488 billion in sales, or 44% of all e-commerce transactions. 

    Data can provide valuable insights, for example, recognizing your high-value customers, their motives behind the purchase, their buying patterns, behaviors, and which are the best channels to market to them and when. Having these detailed insights increases the probability of customer acquisition and perhaps drives their loyalty towards you. 

    7. Pricing

    competitive pricing in retail
    Competitive pricing in retail

    Market trends fluctuate at an unprecedented pace, and pricing has become as competitive as it’s ever been. The only way to keep up with competitive pricing in retail is to use retail analytics that enables retailers to drive more revenue & margin by pricing products competitively

    A report from Inside Big Data found companies experience anywhere from 0.5% up to 17.1% in margin loss purely because of pricing errors. Pricing analytics provides companies with the tools and methods to perceive better, interpret and predict pricing that matches consumer behavior. Appropriate pricing power comes from understanding what your consumers want, which offers they respond to, how and where they shop, and how much they will pay for your products. 

    In 2021, the price optimization segment is anticipated to own the largest share of the overall retail analytics market. Retailers can identify gaps and set alerts to track changes across crucial SKUs or products with pricing analytics. Knowing your customer’s price perception will increase sales and also allow you to design promotions that’ll attract customers. Pricing analytics also accounts for factors like demographics, weather forecasting, inventory levels, real-time sales data, product movement, purchase history, and much more to arrive at an excellent price.  

    8. Sales and Demand Forecasting

    Sales and demand forecasting allow retailers to plan for levels of granularity—monthly, weekly, daily, or even hourly—and use the insights in their marketing campaigns and business decisions. The benefits of a granular forecast are apparent since retailers don’t have to bank on historical data of previous clients and customers to predict revenues. Retailers can plan their strategies and promotions that suit their customer’s demands. 

    With sales and demand forecasting, retailers can also consider the most recent, historical, and real-time data to predict potential future revenue. Sales and demand analytics can predict buying patterns and market trends based on socioeconomic and demographic conditions. 

    9. Customer Service and Experience

    With the development of eCommerce, more and more customers prefer to browse and interact with the product before purchasing online. They look for better deals and discounts across stores and platforms. 3 out of 5 consumers say retail’s investment in technology is improving their online and in-store shopping experiences. To enhance merchandising and marketing strategies, retailers can gather data on customer buying journeys to understand their in-store and online experiences. 

    Retailers can run test campaigns to know the impact on sales and use historical data to predict consumers’ needs based on their demographics, buying patterns, and interests. Retail analytics help retailers to bring more efficiency in promotions and drive impulsive purchases and cross-selling.

    10. Promotion

    Analyze competitors' promotions
    Analyze Competitors’ Promotions

    Promotions are potent sales drivers and need to be cleverly targeted towards specific customers with precise deals to generate outstanding sales. Retail analytics allows companies to study their customers and competitors to a vastly elevated level. 

    To be an industry leader, retail companies not only have to understand their customers, but they must also analyze competitors’ promotions to improve their marketing strategies. Analyzing your competitor’s promotional banners, ads, and marketing campaigns are no more associated with imitation. 

    With data analytics and AI, retailers can watch their competitors’ commercialization strategies. It can uncover vital information about their target audience, sales volume fluctuations, popular seasonal product types, product attributes of popular items, and significant industry trends.  Knowing exactly which products and brands are popular among your competitor’s campaigns can help retailers improve their promotional strategies. 

    Conclusion

    The benefits of retail analytics are spread across various verticals, from merchandising, assortment, inventory management, and marketing to reducing losses. The need for analytics has become even more apparent considering the growing eCommerce platforms, changing customer buying journeys, and the complexity of the industry. Understanding which products sell best among which customers will help retailers to deliver an optimized shopping experience.

    Want to drive profitable growth by making smarter pricing, promotions, and product merchandising decisions using real-time retail insights? DataWeave’s AI-powered Competitive Intelligence can help! Reach out to our Retail Analytics experts to know more.

  • How Artificial Intelligence is giving the  Indian Beauty Industry a Facelift

    How Artificial Intelligence is giving the Indian Beauty Industry a Facelift

    With the help of artificial intelligence and machine learning, beauty and cosmetics companies are exploring new possibilities. According to a report by Avendus, the global beauty and personal care market are expected to touch US$725 billion by 2025 and the young Indian market is expected to grow to $28 billion by then. This segment is a space of opportunity and today we have more than 80 Indian brands in this domain. 

    D2C beauty brand logos
    D2C beauty brand logos

    While technology in this space plays a very important role, Artificial Intelligence (AI) amongst everything else is giving the beauty industry a makeover. This is because, AI can create an impact on all stages of the beauty value chain — from research & development to supply chain management to product selection, marketing, and more! Resonating this thought, Chaitanya Nallan, CEO & Co-Founder, SkinKraft Laboratories mentions “As a digital-first brand, we sell across multiple e-commerce platforms as well as through our own website. Thus, it is very important that we track and maintain inventory across all channels in real-time to avoid stock-outs and loss of sales. We use AI for this. We have built an in-house data tracking dashboard that pulls in inventory information from all warehouses and maps them against sales to give us an accurate estimate of days of inventory across all SKUs and across all platforms. This information directly feeds into our procurement dashboard and also helps the marketing team to create the right sales strategy.”

    Stock availability is crucial to driving sales. If you need help tracking your online inventory – DataWeave can help give you a near real-time view of your product stock status across marketplaces. 

    With AI being a powerful technology wand, here is how it can drive the future of beauty brands within the D2C segment in India.

    Making Virtual Product trials a reality

    virtual product trial
    Virtual Product Trial

    Augmented Reality (AR) is a prevalent term and many companies are already using it on an everyday basis. More commonly, the Snapchat and Instagram filters we use are all powered by AR. In a similar vein, virtual images can be laid over actual images in real-time using AI. And keeping this concept handy, beauty brands are bringing to the front the AR-powered ‘virtual mirrors’ that let consumers try on cosmetic products in real-time. Modiface by L’Oréal is a perfect example of VR-mirrors, which has pioneered the AR-powered makeup try-ons in the market. These virtual mirrors use AI algorithms to detect the user’s face through a camera by focal points and map the face. Then using AR, images of makeup are adjusted according to the terms obtained and overlaid over the features on the face giving consumers a virtual feel of what they’d look like wearing the product. 

    Virtual try on
    Virtual Try-on

    Much recently, Indian brand Lakme has made ‘virtual try on’ possible by creating a smart mirror on its official website that allows customers to watch their reflection, try on different shades, and customize those shades according to their preferences. Shade matching until a few years back was an entirely on-ground phenomenon and customers visiting a local cosmetics store were able to choose and match the shade of compact, eye shadow, and lipstick against their true skin tone. Today AI can allow you to narrow down on products based on a virtual shade card, put them against your skin in real-time. 

    Make it Truly Personalised

    Every customer is unique, and one size does not fit all. Everyone has a personalized beauty regime they follow & understanding this could be the key to success for beauty brands. For this reason, the future of beauty lies in harnessing AI and AR solutions to tailor the beauty shopping experience to match the needs of the individual consumer. This not only enhances digital engagement but also increases purchasing confidence which in turn helps brands drive conversion and brand loyalty.  

    Pre-pandemic, offline beauty advisors played a consultative role when customers were making purchase decisions. A lot of this has moved online – take for instance Olay. It launched an online “Skin Advisor” app based on a deep-learning algorithm that analyses a consumer’s skin using a simple selfie! Armed with information on their skin type, customers can make an informed, personalized purchase that’s right for their specific skin type. 

    Skin Advisor App
    Skin Advisor App

    Understanding customer preferences and using data from their past purchases also help with personalized marketing in a big way. “Data-driven personalization gives brands insight into what their customers are interested in. We integrate this data into our marketing campaigns and deliver specific, personalized, and relevant content. This way, we make sure to target the right audience with the right messaging. This, in turn, helps us increase engagement and retain customers. Moreover, this combined data, allows us to get repeat sales through upselling and cross-selling. Further, knowing customers beyond just simple demographics helps us improve our targeting and helps us predict future behaviour. We’d like to know, for instance, if a customer clicked on our advertisement, liked, or commented on our social media product displays, signed up to our email list, etc. These analytics reveal a customer’s interest. Combine it with demographics – and you get a sense of what the customer is interested in,” Dhruv Madhok, Co-Founder, ARATA highlights. 

    Boost Product Development

    Social listening
    Social listening

    AI algorithms can be used to study and analyse customer feedback. The algorithm works towards interpreting customer comments, reviews, and feedback on a brand’s website, social media channels, and other online platforms. Artificial Intelligence can also decode and analyse questionnaires and feedback forms that the customers may have responded to online or offline. 

    The beauty and personal care industry is largely driven by usage and customer preferences, so gauging how customers feel about key products can help businesses create & develop products that customers will most likely prefer to buy. For instance, reputed beauty brand Avon recently mentioned that it developed the True 5-in-1 Lash Genius Mascara based on actual consumer feedback! They used machine learning & artificial intelligence to read, filter, process & rank thousands of online consumer comments to determine the top features they crave in a mascara. Using this customer gathered intelligence, they developed a unique product that consumers we’re “asking for”!

    True 5-in-1 Lash Genius Mascara by beauty brand Avon
    True 5-in-1 Lash Genius Mascara by beauty brand Avon

    Need help listening to what your consumers are saying about your brand online? Read more about DataWeave’s AI Powered Sentiment Analysis solution.

    More and more brands are listening to customer responses closely to give way to new products, bring in tweaks to their existing basket, and innovate further. “Our ORM team is leading the knowledge accumulation as far as social listening is concerned. They are not just responsible for responding to customer queries, they are also instrumental in highlighting key insights based on user behaviour being observed,” Chaitanya of SkinKraft Laboratories further asserts. 

    Bombay Shaving Company too with its data-centric culture leverages customer responses for decision making & product development. “In-home personal care and hygiene exploded during the pandemic. We used data analytics to explore different dimensions of in-home experience-driven needs (new usage occasions, need for convenience and DIY, etc.). We listened to our customers & were able to introduce our women’s brand, with innovative hair removal products in a big way during this period. Which today contributes to a significant percentage of our business,” Shantanu Deshpande, Founder & CEO, Bombay Shaving Company mentions.

    Given the scope and scale of the beauty and personal care industry that is major ‘usage’ driven, Artificial Intelligence with its diverse potential can bring a paradigm shift in the industry. AI can help not only with virtual trials, personalization, listening in to customers’ feedback but also with monitoring a brand’s Digital Shelf. Brands can amplify their online sales by tracking Digital Shelf KPIs like share of search & product visibility, pricing & discounting, product content, availability & assortment. Reach out to our Digital Shelf experts to learn more.

  • AI-Driven Mapping of Retail Taxonomies- Part 2

    AI-Driven Mapping of Retail Taxonomies- Part 2

    Mapping product taxonomies using Deep Learning

    In Part 1 we discussed the importance of Retail taxonomy and the applications of mapping retail taxonomies in Assortment Analytics, building Knowledge Graph, etc. Here, we will discuss how we approached the problem of mapping retail taxonomies across sources.

    We solved this problem by classifying every retail product to a standard DataWeave defined taxonomy so that products from different websites could be brought at the same level. Once these products are at the same level, mapping taxonomies becomes straightforward.

    We’ve built an AI-based solution that uses state-of-the-art algorithms to predict the correct DataWeave Taxonomy for a product from its textual information like Title, Taxonomy and Description. Our model predicts a standard 4 level (L1-L2-L3-L4) taxonomy for any given product. These Levels denote Category, Sub Category, Parent Product Type and Product Type respectively.

    Approach

    Conventional methods for taxonomy prediction are typically based on machine learning classification algorithms. Here, we need to provide textual data and the classifier will predict the entire taxonomy as a class.

    We used the classification approach as a baseline, but found a few inherent flaws in this:

    • A Classification model cannot understand the semantic relation between input text and output hierarchy. Which means, it cannot understand if there’s any relation between the textual input and the text present in the taxonomy. For a classifier, the output class is just a label encoded value
    • Since the taxonomy is a tree and each leaf node uniquely defines a path from the root to leaf, the classification algorithms effectively output an existing root-to-leaf path. However, it cannot predict new relationships in the tree structure
    • Let’s say, our training set has only the records for “Clothing, Shoes & Jewelry > Men > Clothing > Shorts” and  “Clothing, Shoes & Jewelry > Baby > Shoes > Boots”, Example:

    {‘title’: “Russell Athletic Men’s Cotton Baseline Short with Pockets – Black – XXX-Large”, 

    ‘dw_taxonomy’: “ Clothing, Shoes & Jewelry > Men > Clothing > Shorts”},

    {‘title’:” Surprise by Stride Rite Baby Boys Branly Faux-Leather Ankle Boots(Infant/Toddler) – Brown -”,

    ’dw_taxonomy:” Clothing, Shoes & Jewelry > Baby > Shoes > Boots”}

    Now, if a product with Title “Burt’s Bees Baby Baby Boys’ Terry Short” comes for prediction, then the classifier will never be able to predict the correct taxonomy. Although, it would have seen the data points of Shorts and Baby.

    E-commerce product taxonomy has a very long tail, i.e. there’s a huge imbalance in counts of data per taxonomy. Classification algorithms do not perform well for very long tail problems.

    Encoder-Decoder with Attention for Taxonomy Classification

    What is Encoder-Decoder?

    Encoder-Decoder is a classical Deep Learning architecture where there are two Deep Neural Nets, an Encoder and a Decoder linked with each other to generate desired outputs.

    The objective of an Encoder is to encode the required information from the input data and store it in a feature vector. In case of text input, the encoder is mostly an RNN or Transformer based architecture and for image input, it is mostly a CNN-based architecture. Once the encoded feature vector is created, the Decoder uses it to produce the required output. The Encoder and Decoder can be interfaced by another layer which is called Attention. The Role of Attention mechanism is to train the model to selectively focus on useful parts of the input data and hence, learn the alignment between them. This helps the model to cope effectively with long input sentences (when dealing with text) or complex portions of images (when input is an image).

    Instead of classification-based approaches, we use an Encoder-Decoder architecture and map the problem of taxonomy classification to the task of machine translation (MT) AKA, Seq2Seq. An MT system takes the text in one language as input and outputs its translation as a sequence of words in another language. In our case, the input maps to the textual description of a product, and the output maps to the sequence of categories and sub-categories in our taxonomy (e.g., Clothing, Shoes & Jewelry > Baby > Shoes > Boots). By framing taxonomy classification as an MT problem, we overcome a lot of limitations present in classical classification approaches.

    • This architecture has the capability to predict a taxonomy that is not even present in the training data.
      • Talking about the example we discussed earlier where a traditional classification model was not able to predict the taxonomy for “Baby Boys knit terry shorts – cat & jack gray 12 m”, this Encoder-decoder model easily predicts the correct taxonomy as “ Clothing, Shoes & Jewelry > Baby > Clothing > Shorts”
    • We achieved a much higher accuracy because the model understands the semantic relationship between the input and output text, as well as giving attention to the most relevant parts in the input, when generating the output
    Fig. Attention visualization for product title “South of France lavender fields Bar Soap”. It can be seen from the image that the attention weights of “soap” word is very high when predicting the output at different time-steps.

    We used pre-trained fasttext word embeddings to vectorize textual input, pass on to the GRU-RNN based encoder which processes the input sequentially, and generates the final encoded vector. The Decoder which is also a GRU-RNN takes this encoded input and generates the output sequentially. Along with the encoded vector, there is also an attention vector which is passed to the Decoder for the output at every time-step.

    We trained both the Classification model (Baseline) and the Encoder-Decoder model for the Fashion category and the Beauty & Personal Care category. 

    For Fashion, we trained the model with 170,000 data points and validated it on a 30k set. For Beauty Category, we trained the model on 88k data points and validated it on a 20k set. We were able to achieve 92% Seq2Seq accuracy in 1,240 classes for the Fashion category and 96% Seq2Seq accuracy in 343 classes for the Beauty Category, using the Encoder-Decoder approach.

    Summary and the Way Forward

    Since we moved to this approach, we have seen drastic improvements in the accuracy of our Assortment Intelligence accounts. But the road doesn’t end here. There are several challenges to be tackled and worked upon. We’re planning on making this process language agnostic by using cross-lingual embeddings, merging models from different categories and also using product Image to complement the text-based model with visual input via a Multi-Modal approach.

    References

    Don’t Classify, Translate: Multi-Level E-Commerce Product Categorization Via Machine Translation by Maggie Yundi Li, Stanley Kok and Liling Tan

    SIGIR eCom’18 Data Challenge is organized by Rakuten Institute of Technology Boston (RIT-Boston)

    Massive Exploration of Neural Machine Translation Architectures by Denny Britz, Anna Goldie, Minh-Thang, and Luong Quoc Le

  • Compete Profitably in Retail: Leveraging AI-Powered Competitive Intelligence at Massive Scale

    Compete Profitably in Retail: Leveraging AI-Powered Competitive Intelligence at Massive Scale

    AI is everywhere. Any retailer worth his salt knows that in today’s hyper-competitive environment, you can’t win just by fighting hard – you have to do it by fighting smart. The solution? Retailers are turning to AI in droves.

    The problem is that many organizations regard AI as a black box of sorts – where you can throw all your data (the digital era’s blessing that feels like a curse) in at one end and have miraculously meaningful output appearing out the other. The reality of how AI works, however, is a lot more complex. It takes a lot of work to make AI work for you – and then to derive value out of it.

    Image Source: https://xkcd.com/1838

    Following the advent of the digital era, businesses across industries, particularly retail, were left grappling with massive amounts of internal data. To make things worse, this data was unstructured and siloed, making it difficult to process effectively. Yet, businesses learned to leverage simple analytics to extract relevant data and insights to affect smarter decisions.

    But just as that happened, the e-commerce revolution stirred things up again. As businesses of all shapes, sizes, and types moved online, they suddenly became a whole lot more vulnerable to other players’ movements than they were just about a decade ago, when buyers rarely visited more than one store before they made a purchase. In other words, retailers are now operating in entire ecosystems – with consumers evaluating a number of retailers before making a purchase, and a disproportionate number of players vying for the same consumer mindshare and share of wallet.

    Thus, external data from the web – the largest source of data known to man at present – is becoming critical to business’ ability to compete profitably in the market.

    Competing profitably in the digital era: Can AI help?

    As organizations across industries and geographies increasingly realized that their business decisions were affected by what’s happening around them (such as competitors’ pricing and merchandize decisions), they started shifting away from their excessive obsession with internal data, and began to look for ways to gather external data, integrate it with their internal data, and process it all in entirety to derive wholesome, meaningful insights.

    Simply put, harnessing external data consistently and on a large scale is the only way for businesses to gain a sustainable competitive advantage in the retail market. And the only way to practically accomplish that is with the help of AI. Many global giants are already doing this – they’re analyzing loads of external data every minute to take smarter decisions.

    That said, though, what you need to know is that all this data, while publicly available and therefore accessible, is massive, unstructured, noisy, scattered, dynamic, and incomplete. There’s no algorithm in the world that can start working on it overnight to churn out valuable insights. AI can only be effective if enormous amounts of training data is constantly fed back into it, coaxing it to get better and more astute each time. However, given the scarcity of readily available training datasets, limited and unreliable access to domain-specific data, and the inconsistent nature of the data itself, a majority of AI initiatives have ended up in a “garbage in, garbage out” loop that they can’t break out of.

    What you need is the perfect storm

    At DataWeave, we understand the challenge of blindly dealing with data at such a daunting scale. We get that what you need is a practical way to apply AI to the abundant web data out there and generate specific, relevant, and actionable insights that enable you to make the right decisions at the right time. That’s why we’ve developed a system that runs on a human-aided-machine-intelligence driven virtuous loop, ensuring better, sharper outcomes each time.

    Our technology platform includes four modules:

    1. Data aggregation: Here, we capture public web data at scale – whatever format, size, or shape it’s in – by deploying a variety of techniques.

    2. AI-driven analytics: Since the gathered data is extremely raw, it’s cleaned, curated, and normalized to remove the noise and prepare it for the AI layer, which then analyzes the data and generates insights.

    3. Human-supervised feedback: Though AI is getting smarter with time, we see that it’s still far from human cognitive capabilities – so we’ve introduced a human in the loop to validate the AI-generated insights, and use this as training data that gets fed back to the AI layer. Essentially, we use human intelligence to make AI smarter.

    4. Data-driven decision-making: Once the data has been analyzed and the insights generated, they can either be used as it to drive decision-making, or then integrated with internal data for decision-making at a higher level.

    With intelligent, data-backed decision-making capabilities, you can outperform your competitors

    Understandably, pricing is one of the most popular applications of data analytics in retail. For instance, a leading, US-based online furniture retailer approached us with the mission-critical challenge of pricing products just right to maximize sell-through rates as well as gross margin in a cost-effective and sustainable manner. We matched about 2.5 million SKUs across 75 competitor websites using AI and captured pricing, discounts, and stock status data every day. As a result, we were able to affect an up to 30% average increase in the sales of the products tracked, and up to a 3x increase in their gross margin.

    DataWeave’s powerful AI-driven platform is essentially an engine that can help you aggregate and process external data at scale and in near-real time to manage unavoidably high competition and margin pressures by enabling much sharper business decisions than before. The potential applications for the resulting insights are diverse – ranging from pricing, merchandize optimization, determination of customer perception, brand governance, and business performance analysis.

    If you’d like to learn more about our unique approach to AI-driven competitive intelligence in retail, reach out to us for a demo today!

  • Evaluating the Influence of Learning Models

    Evaluating the Influence of Learning Models

    Natt Fry, a renowned thought leader in the world of retail and analytics, published recently an article expounding the value and potential of learning models influencing business decision-making across industries over the next few years.

    He quotes a Wall Street Journal article (paywall) published by Steven A. Cohen and Matthew W. Granade who claim that, “while software ate the world the past 7 years, learning models will ‘eat the world’ in the next 7 years.”

    The article defines a learning model as a “decision framework in which the logic is derived by algorithm from data. Once created, a model can learn from its successes and failures with speed and sophistication that humans usually cannot match.”

    Narrowing this down to the world of retail, Natt states, “if we believe that learning models are the future, then retailers will need to rapidly transform from human-learning models to automated-learning models.”

    This, of course, comes with several challenges, one of which is the scarcity of easily consumable data for supervised learning algorithms to get trained on. This scarcity often results in a garbage-in-garbage-out situation and limits the ability of AI systems to improve in accuracy over time, or to generate meaningful output on a consistent basis.

    Enabling Retailers Become More Model-Driven
    As a provider of Competitive Intelligence as a Service to retailers and consumer brands, DataWeave uses highly trained AI models to harness and analyze massive volumes of Web data consistently.

    Far too often, we’ve seen traditional retailers rely disproportionately on internal data (such as POS data, inventory data, traffic data, etc.) to inform their decision-making process. This isn’t a surprise, as internal data is readily accessible and likely to be well structured.

    However, if retailers can harness external data at scale (from the Web — the largest and richest source of information, ever), and use it to generate model-driven insights, they can achieve a uniquely holistic perspective to business decision-making. Also, due simply to the sheer vastness of Web data, it serves as a never-ending source of training data for existing models.

    DataWeave’s AI-based model to leverage Web data

     

    Web data is typically massive, noisy, unstructured, and constantly changing. Therefore, at DataWeave, we’ve designed a proprietary data aggregation platform that is capable of capturing millions of data points from complex Web and mobile app environments each day.

    We then apply AI/ML techniques to process the data into a form that can be easily interpreted and acted on. The human-in-the-loop is an additional layer to this stack which ensures a minimum threshold of output accuracy. Simultaneously, this approach feeds information on human-driven decisions back to the algorithm, thereby rendering it more and more accurate with time.

    Businesses derive the greatest value when external model-based competitive and market insights are blended with internal data and systems to generate optimized recommendations. For example, our retail customers combine competitor pricing insights provided by our platform with their internal sales and inventory data to develop algorithmic price optimization systems that maximize revenue and margin for millions of products.

    This way, DataWeave enables retailers and consumer brands to utilize a unique model-based decision framework, something that will soon be fundamental (if not already) to business decision-making across industry verticals and global regions.

    As AI-based technologies become more pervasive in retail, it’s only a matter of time before they’re considered merely table stakes. As summarized by Natt, “going forward, retailers will be valued on the completeness of the data they create and have access to.”

    If you would like to learn more about how we use AI to empower retailers and consumer brands to compete profitably, check out our website!

    Read Natt’s article in full below:

    Steven A. Cohen and Matthew W. Granade published a very interesting article in the Wall Street Journal on August 19, 2018 — https://www.wsj.com/articles/models-will-run-the-world-1534716720

    Their premise is that while software ate the world (Mark Andreessen essay in 2011, “Why Software is Eating the World”) the past 7 years, learning models will “eat the world” in the next 7 years.

    A learning model is a decision framework in which the logic is derived by algorithm from data. Once created, a model can learn from its successes and failures with speed and sophistication that humans usually cannot match.

    The authors believe a new, more powerful, business opportunity has evolved from software. It is where companies structure their business processes to put continuously learning models at their center.

    Amazon, Alibaba, and Tencent are great examples of companies that widely use learning models to outperform their competitors.

    The implications of a model-driven world are significant for retailers.

    Incumbents can have an advantage in a model-driven world as they already have troves of data.

    Going forward retailers will be valued on the completeness of the data they create and have access to.

    Retailers currently rely on the experience and expertise of their people to make good decisions (what to buy, how much to buy, where to put it, etc.).

    If we believe that learning models are the future then retailers will need to rapidly transform from human-learning models to automated-learning models, creating two significant challenges.

    First, retailers have difficulty in finding and retaining top learning-model talent (data scientists).

    Second, migrating from human-based learning models to machine-based learning models will create significant cultural and change management issues.

    Overcoming these issues is possible, just as many retailers have overcome the issues presented by the digital age. The difference is, that while the digital age has developed over a 20 year period, the learning-model age will develop over the next 7 years. The effort and pace of change will need to be much greater.

  • Recognize Product Attributes with AI-Powered Image Analytics

    Recognize Product Attributes with AI-Powered Image Analytics

    Anna is a fashionista and a merchandise manager at a large fast-fashion retailer. As part of her job, she regularly browses through the Web for the most popular designs and trends in contemporary fashion, so she can augment her product assortment with fresh and fast-moving products.

    She spots a picture on social media of a fashion blogger sporting a mustard colored, full-sleeved, woolen coat, a yellow sweatshirt, purple polyester leggings, and a pair of pink sneakers with laces. She finds that the picture has garnered several thousand “likes” and several hundred “shares”. She also sees that a few other online fashion influencers have blogged about similar styles in coats and shoes being in vogue.

    Anna thinks it’s a good idea to house a selection of similar clothing and accessories for the next few weeks, before the trend dies down.

    But, she is in a bit of a pickle.

    Different brands represent their catalog differently. Some have only minimalistic text-based product categorization, while others are more detailed. The ones that are detailed don’t categorize products in a way that helps her narrow down her consideration set. Product images, too, lack standardization as each brand has its own visual merchandising norms and practices.

    Poring through thousands of products across hundreds of brands, looking for similar products is time-consuming and debilitating for Anna, restricting her ability to spend time on higher-value activities. Luckily, at DataWeave, we’ve come across several merchandise managers facing challenges like hers, and we can help.

    AI-powered product attribute tagging in fashion

    DataWeave’s AI-powered, purpose-built Fashion Tagger automatically assigns labels to attributes of fashion products at great granularity. For example, on processing the image of the blogger described earlier, our algorithm generated the following output.

    Original Image Source: Rockpaperdresses.dk

    Vision beyond the obvious

    Training machines is hard. While modern computers can “see” as well as any human, the difference lies in their lack of ability to perceive or interpret what they see.

    This can be compared to a philistine at a modern art gallery. While he or she could quite easily identify the colors and shapes in the paintings, additional instructions would be needed on how the painting can be interpreted, evaluated, and appreciated.

    While machines haven’t gotten that far yet, our image analytics platform is highly advanced, capable of identifying and interpreting complex patterns and attributes in images of clothing and fashion accessories. Our machines recognize various fashion attributes by processing both image- and associated text-based information available for a product.

    Here’s how it’s done:

    • With a single glance of its surroundings, the human eye can identify and localize each object within its field of view. We train our machines to mimic this capability using neural-network-based object detection and segmentation. As a result, our system is sensitive to varied backgrounds, human poses, skin exposure levels, and more, which are quite common for images in fashion retail.
    • The image is then converted to 0s and 1s, and fed into our home-brewed convolutional neural network trained on millions of images with several variations. These images were acquired from diverse sources on the Web, such as user-generated content (UGC), social media, fashion shows, and hundreds of eCommerce websites around the world.
    • If present, text-based information associated with images, like product title, metadata, and product descriptions are used to enhance the accuracy of the output and leverage non-visual cues for the product, like the type of fabric. Natural-language processing, normalization and several other text processing techniques are applied here. In these scenarios, the text and image pipelines are merged based on assigned weightages and priorities to generate the final list of product attributes.

    The Technology Pipeline

    Our Fashion Tagger can process most clothing types in fashion retail, including casual wear, sportswear, footwear, bags, sunglasses and other accessories. The complete catalog of clothing types we support is indicated in the image below.

    Product Types Processed and Classified by DataWeave

    One product, several solutions

    Across the globe, our customers in fast-fashion wield our technology every day to compare their product assortment against their competitors. Our SaaS-based portal provides highly granular product-attribute-wise comparisons and tracking of competitors’ products, enabling our customers to spot assortment gaps of in-demand and trending products, as well as to better capitalize on the strengths in their assortment.

     

    Some other popular use cases include:

    • Similar product recommendations: This intelligent product recommendation engine can help retailers identify and recommend to their shoppers, products with similar attributes to the one they’re looking at, which can potentially help drive higher sales. For example, they can recommend alternatives to out-of-stock products, so customers don’t bounce off their website easily.
    • Ensemble recommendations: Our proprietary machine-learning based algorithms analyze images on credible fashion blogs and websites to learn the trendiest combinations of products worn by online influencers, helping retailers recommend complementary products and drive more value. Combining this with insights on customer behavior can generate personalized ensemble recommendations. It’s almost like providing a personal stylist for shoppers!
    • Diverse styling options: The same outfit can often be worn in several different ways, and shoppers typically like to experiment with unconventional modes of styling. Our technology helps retailers create “lookbooks” that provide real world examples of multiple ways a particular piece of clothing can be worn, adding another layer to the customer’s shopping experience.
    • Search by image: Shoppers can search for products similar to ones worn by celebrities and other influencers through an option to “Search by Image”, which is possible due to our technology’s ability to automatically identify product attributes and find similar matches.
    • Fast-fashion trend analysis: Retailers can study emerging trends in fashion and host them in their product assortment before anyone else.

    The devil is in the details

    DataWeave’s Fashion Tagger guarantees very high levels of accuracy. Our unique human-in-the-loop approach combines the power of machine-learning-based algorithms with human intelligence to accurately differentiate between similar product attributes, such as between boat, scoop and round necks in T-shirts.

    This system is a closed feedback loop, in which a large amount of ground-truth (manually verified) data is generated by in-house teams, which power the algorithms. In this way, the machine-generated output gets more and more accurate with time, which goes a long way in our ability to swiftly deliver insights at massive scale.

    In summary, DataWeave’s Image Analytics platform is driven by: enormous amount of training data + algorithms + infrastructure + humans-in-loop.

    If you’re intrigued by DataWeave’s technology and wish to know more about how we help fashion retailers compete more effectively, check us out on our website!

     

  • Alibaba’s Singles Day Sale: Decoding the World’s Biggest Shopping Festival

    Alibaba’s Singles Day Sale: Decoding the World’s Biggest Shopping Festival

    $17.5 million every 60 seconds.

    That’s the volume of sales Alibaba generated on 11.11, or Singles Day. This mammoth event, decisively the world’s biggest shopping day, dwarfed last years’ Black Friday and Cyber Monday combined.

    This year, the anticipation around Singles Day was all-pervasive, and the sale was widely expected to break all records, as more than 60,000 global brands queued up to participate. By the end of the day, sales topped $25.3 billion, while shattering last year’s record by lunchtime.

    It’s an astonishing feat of retailing, eight years in the making. When Alibaba first started 11.11 in 2009, they set out strategically to try and convert shopping into a sport, infusing it with a strong element of entertainment. “Retail as entertainment” is a unique central theme for 11.11 and this year Alibaba leveraged its media and eCommerce platforms in concert to create an entirely immersive experience for viewers and consumers alike.

    From a technology perspective, the “See Now, Buy Now” fashion show and the pre-sale gala seamlessly merged offline and online shopping so viewers tuning in to both shows can watch them while simultaneously shopping via their phones or saving the items for a later date.

    The eCommerce giant also collaborated with roughly 50 shopping malls in China to set up pop-up shops, eventually extending its shopper reach to span 12 cities.

    Of course, attractive discounts on its eCommerce platforms were on offer as well.

    Deciphering Taobao.com

    At DataWeave, we have been analyzing the major sale events of several eCommerce companies from around the world. During Singles Day, when we trained our data aggregation and analysis platform on Taobao.com (Alibaba’s B2C eCommerce arm), and its competitors JD.com and Amazon.ch, our technology platform and analysts had to overcome two primary challenges:

    1. All text on these websites were in Chinese

    All information — names of products, brands, and categories — were displayed in Chinese. However, our technology platform is truly language agnostic, capable of processing data drawn from websites featuring all international languages. Several of our customers have benefited strategically from this unique capability.

    2.  Discounted prices were embedded in images on Taobao.com

    While it’s normal for sale prices to be represented in text on a website (relatively easy to capture by our advanced data aggregation system), Taobao chose to display these prices as part of its product images — like the one shown in the adjacent image.

    However, our technology stack comprises of an AI-powered, state-of-the-art image processing and analytics platform, which quickly extracted the selling prices embedded in the images at very high accuracy.

    We analyzed the Top 150 ranked products of over 20 product types , spread across Electronics, Men’s Fashion, and Women’s Fashion, representing over 25,000 products in total, each day, between 8.11 and 12.11.

    In the following infographic, we analyze the absolute discounts offered by Taobao on 11.11, compared to 8.11 (based on pricing information extracted from the product images using our image analytics platform), together with an insight into the level of premium products included in their mix for each product type, between the two days of comparison.

    Unexpectedly, we noticed that each day, ALL the products in the Top 150 ranks differed from the previous day — a highly unique insight into Taobao’s unique assortment strategy.

    Counter-intuitively, absolute discounts across all categories were considerably higher on 8.11 than on 11.11, even if it were for a marginally fewer number of products. The number of discounted Electronics products on sale rose on 11.11 compared to 8.11 (124 versus 102 respectively), while there was little movement in the number of discounted Men’s Fashion(55 versus 57) and Women’s Fashion (35 verses 27) products.

    Taobao targeted the mobile phone and tablets segment with aggressive discounts (21.0 percent and 18.2 percent respectively), compared to the average Electronics discount level of 7.7 percent.

    Interestingly, the average selling price drifted up for Electronics on 11.11 compared to 8.11 (¥4040 versus ¥3330). Men’s Fashion dropped to ¥584 from ¥604 while prices for Women’s Fashion was stable.

    It’s clear that even with all the fanfare, Singles Day didn’t produce the level of discounts that one might have expected, indicating that purchases were driven as much by the hype surrounding the event as anything else.

    How did Alibaba’s Competitors Fare?

    While Taobao was widely expected to offer discounts during Alibaba’s major sale event, we looked at how its competitors JD.com and Amazon.ch reacted to Taobao’s strategy.

    As over 80 percent of top-ranked products were consistently present in the Top 150 ranks of each product type on these websites, we analyzed the additional discounts offered during 11.11, compared to prices on 8.11.

    Broadly speaking, both Amazon.ch and JD.com appear to have elected not to go head to head with Taobao on specific segments. JD.com’s discount strategy was spearheaded by Sports Shoes (22.1 percent) and Refrigerators (14.8 percent) while Amazon.ch featured TVs (15.3 percent) and Mobile Phones (10.2 percent).

    The average additional discounts offered by Amazon.ch and JD.com in Electronics (8.4 percent) was slightly above Taobao’s overall absolute discount (7.7 percent). TCL was aggressive with its pricing on both websites, offering over 20% discount on almost its entire assortment.

    Surprisingly, JD.com swamped Amazon.ch’s number of additionally discounted products, across all three featured categories although this may be partially explained by Amazon.ch electing to adopt a significantly more premium price position in both Men’s and Women’s Fashions compared to JD.com, while remaining roughly line ball on Electronics.

    Jack Ma’s “New Retail”

    Interestingly, JD.com wasn’t far behind Taobao in terms of sales, clocking up $20 billion in revenue, and sparking an interesting public debate between the two eCommerce giants extolling their respective performances.

    Singles Day is one of the pillars of Jack Ma’s vision of a “New Retail” represented by the merging of entertainment and consumption. Ma’s vision sees the boundary between offline and online commerce disappearing as the focus shifts dramatically to fulfilling the personalized needs of individual customers.

    Hence, Alibaba’s Global Shopping Festival should be understood as not just a one-day event that produces massive revenue, but as a demonstrable tour de force of Alibaba’s vision for the future of retail. One thing is certain — as competition heats up between Chinese retailers, we can be prepared for another Singles Day shoot-out sale next year that one-ups the staggering sales volumes this year.

    If you’re intrigued by DataWeave’s technology, check out our website to learn more about how we provide Competitive Intelligence as a Service to retailers and consumer brands globally.

     

  • Video: Using Product Images to Achieve Over 90% Accuracy in Matching E-Commerce Products

    Video: Using Product Images to Achieve Over 90% Accuracy in Matching E-Commerce Products

    Matching images is hard!

    Images, intrinsically, are complex forms of information, with varying backgrounds, orientations, and noise. Developing a reliable system that achieves human-like accuracy in identifying, interpreting, and comparing images, without investing in expensive resources, is no mean task.

    For DataWeave, however, the ability to accurately match images is fundamental to the value we provide to retailers and consumer brands.

    Why Match Images?

    Our customers rely on us for timely and actionable insights on their competitors’ pricing, assortment, promotions, etc. compared to their own. To enable this, we need to identify and match products across multiple websites, at very large scale.

    One might hope to easily match products using just the product titles and descriptions on websites. However, therein lies the rub. Text-based fields are typically unstructured, and lack consistency or standardization across websites (especially for fashion products). In the following example, the same Adidas jacket is listed as “Tiro Warm-Up Jacket, Big Boys (8–20)” on Macy’s and “Youth Soccer Tiro 15 Training Jacket” on Amazon.

    Hence, instead of using text-based information, we considered using deep-learning techniques to match the images of products listed on e-commerce websites. This, though, requires massive GPU resources and training data fed into the deep-learning model — an expensive proposition.

    The solution we arrived upon, was to complement our image-matching system with the text-based information available in product titles and descriptions. Analyzing this combination of both text- and image-based information enabled us to efficiently match products at greater than 90% accuracy.

    How We Did It

    A couple of weeks ago, I gave a talk at Fifth Elephant, one of India’s renowned data science conferences. In the talk, I demonstrated DataWeave’s innovation of augmenting the NLP capabilities of Solr (a popular text search engine) with deep-learning features to match images with high accuracy.

    Check out the video of the presentation for a detailed account of the system we built:

    Human-Aided Machine Intelligence

    All products matched with the seed product are tagged with a corresponding confidence score. When this score crosses a certain threshold, it’s presumed to be a direct match. The ones that are part of a lower range of confidence scores are quickly examined manually for possible direct matches.

    The outcome, therefore, is that our technology narrows down the consideration set of possible product matches from a theoretical upper limit of millions of products, to only a few tens of products, which are then manually checked. This unique approach has two distinct advantages:

    • The human-in-the-loop enables us to achieve greater than 90% accuracy in matching millions of products — a key differentiator.
    • Information on all manually matched products is continually fed to the deep-learning model, which is used as training data, further enhancing the accuracy of the product matching mechanism. As a result, both our accuracy and delivery time keep improving with time.

    As the world of online commerce continues to evolve and becomes more competitive, retailers and consumer brands need the ability to make quick proactive and reactive decisions, if they are to stay competitive. By building an automated self-improving system that matches products quickly and accurately, DataWeave enables just that.

    Find out more about how retailers and consumer brands use DataWeave to better understand their competitive environment, optimize customer experience, and drive profitable growth.

  • Baahubali 2: Dissecting 75,000 Tweets to Uncover Audience Sentiments

    Baahubali 2: Dissecting 75,000 Tweets to Uncover Audience Sentiments

    Why did Katappa kill Baahubali?

    Two years ago, not many would have foreseen this sentence capturing the imagination of the country like it has. Demolishing all regional barriers, the movie has grossed over INR 500 crores across the world in only its first three days.

    While the first movie received lavish praise for its ambition, technical values, and story, the sequel, bogged by bloated expectations, has polarized the critics fraternity. Some critics compare the movie’s computer graphics favorably to Hollywood productions like Lord of the Rings. Others find the movie lacking in pacing and plot.

    The masses, however, have reportedly lapped the movie up. Social media channels are brimming with opinions, and if one is to attempt finding out the aggregate views of audiences, Twitter is a good place to start.

    At DataWeave, we ran our proprietary, AI-powered ‘Sentiment Analysis’ algorithm over all tweets about Baahubali 2 the first three days of its release, and observed some interesting insights.

    Twitterati Reactions to Baahubali 2

    Overall, the Twitterati’s views on the movie were overwhelmingly positive. We analysed over 75,000 tweets and identified the sentiments expressed on several facets of the movie, such as, Visuals, Acting, Prabhas, etc. The following graphic indicates how the movie fared in some of these categories.

    The Baahubali team, Anushka (actor), Rajamouli (director), and Prabhas (actor), are all perceived as huge positive influences on the movie. Rajamouli, specifically, met with almost universal approval for his dedication and execution. Several viewers cheered the movie on as a triumph of Indian cinema, one which has redefined the cinema landscape of the country. There was considerable praise for the story, Rana (actor), and acting performances, as well.

    The not-so-positive sentiments were reserved for the reason behind Katappa killing Baahubali (no spoilers!), the visuals, and the second half of the movie. Many viewers found the second half to be slow, with unrealistic visuals and action sequences. For example, one of the tweets read:

    “First half was good, but the second half is beyond Rajnikanth movies: humans uprooting trees!”

    While these insights seem simple enough to understand, the technology to filter inevitably chaotic online content and extract meaningful information is incredibly complex.

    Unearthing Meaning from Chaos

    At DataWeave, we provide enterprises with Competitive Intelligence as a Service by aggregating and analyzing millions of unstructured data points on the web, across multiple sources. This enables businesses to better understand their competitive environment and make data-driven decisions to grow their business.

    One of our solutions — Sentiment Analysis — helps brands study customer preferences at a product attribute level by analyzing customer reviews. We used the same technology to analyze the reaction of audiences globally to Baahubali 2. After data acquisition, this process consists of three steps –

    Step 1: Features Extraction

    To identify the “features” that reviewers are talking about, we first understand the syntactical structure of the tweets and separate words into nouns, verbs, adjectives, etc. This needs to account for complexities like synonyms, spelling errors, paraphrases, noise, etc. Our AI-based technology platform then uses various advanced techniques to generate a list of “uni-features” and “compound features” (more than one word for a feature).

    Step 2: Identifying Feature-Opinion Pairs

    Next, we identify the relationship between the feature and the opinion. One of the reasons this is challenging with twitter is, most of the time, twitter users treat grammar with utter disdain. Case in point:

    “I saw the movie visuals awesome bad climax felt director unnecessarily dragged the second half”

    In this case, the feature-opinion pairs are visuals: awesome, climax: bad, second half: unnecessarily dragged. Clearly, something as simple as attributing the nearest opinion-word to the feature is not good enough. Here again, we use advanced AI-based techniques to accurately classify feature-opinion pairs.

    We classified close to 1000 opinion words and matched them to each feature. The infographic below shows groups of similar words that the AI algorithm clustered into a single feature, and the top positive and negative sentiments expressed by the Twitterati for each feature.

    While our technology can associate words with similar meaning, such as, ‘part after interval’ and ‘second half’, it can also identify spelling errors by identifying and grouping ‘Rajamouli’ and ‘Raajamouli’ as a single feature.

    Adjectives like ‘magnificent’ and ‘creative’ were used to describe the Baahubali team positively, while words like ‘boring’, ‘disappointed’, and ‘tiring’ were used to describe the second half of the movie negatively.

    Step 3: Sentiment Calculation

    Lastly, we calculate the sentiment score, which is determined by the strength of the opinion-word, number of retweets and the time of tweet. A weighted average is normalized and we generate a score on a scale of 0% to 100%.

    A Peephole into the Consumer’s Mind

    As more and more people express their views and opinions in the online world, there is more of an opportunity to use these data points to drive business strategies.

    Consumer-focused brands use DataWeave’s Sentiment Analysis solution as a key element of their product strategy, by reinforcing attributes with positive sentiments in reviews, and improving or eliminating attributes with negative sentiments in reviews.

    Click here to find out more about the benefits of using DataWeave’s Sentiment Analysis!

     

  • Why is Product Matching Difficult? | DataWeave

    Why is Product Matching Difficult? | DataWeave

    Product Matching is a combination of algorithmic and manual techniques to recognize and match identical products from different sources. Product matching is at the core of competitive intelligence for retail. A competitive intelligence product is most useful when it can accurately match products of a wide range of categories in a timely manner, and at scale.

    Shown below is PriceWeave’s Products Tracking Interface, one of the features where product matching is in action. The Products Tracking Interface lets a brand or a retailer track their products and monitor prices, availability offers, discounts, variants, and SLAs on a daily (or a more frequent) basis.

     

    A snapshot of products tracked for a large online mass merchant

     

    Expanded view for a product shows the prices related data points from competing stores

    Product Matching helps a retailer or a brand in several ways:

    • Tracking competitor prices and stock availability
    • Organizing seller listings on a marketplace platform
    • Discovering gaps in product catalog
    • Filling the missing attributes in product catalog information
    • Comparing product life cycles across competitors

    Given its criticality, every competitive intelligence product strives hard to make its product matching accurate and comprehensive. It is a hard problem, and one that cannot be complete addressed in an automated fashion. In the rest of this post, we will talk about why product matching is hard.

    Product Matching Guidelines

    Amazon provides a guideline to sellers about how they should write product catalog information in order to achieve a good product matching with respect to their seller listings. These guidelines apply to any retail store or marketplace platform. The trouble is, more often than not these guidelines are not followed, or cannot by retailers because they don’t have access to all the product related information. Some of the challenges are:

    • Products either don’t have a UPC code or it is not available. There are also non-standard products, unbranded products, and private label products.
    • There are products with slights variations in technical specifications, but the complete specs are not available.
    • Retailers manage a huge catalog of accessories, for instance Electronics Accessories (screen guards, flip covers, fancy USB drives, etc.).
    • Apparels and Lifestyle products often have very little by way of unique identifiers. There is no standard nomenclature for colors, material and style.
    • Products are often bundled with accessories or other related products. There are no standard ways of doing product bundling.

    In the absence of standard ways of representing products, every retailer uses their own internal product IDs, product descriptions, and attribute names.

    Algorithmic Product Matching using “Document Clustering”

    Algorithmic product matching is done using some Machine Learning, typically techniques from Document Clustering. A document is a text document or a web page, or a set of terms that usually occur within a “context”. Document clustering is the process of bringing together (forming clusters of) similar documents, and separating our dissimilar ones. There are many ways of defining similarity of documents that we will not delve into in this post. Documents have “features” that act as “identifiers” that help an algorithm cluster them.

    A document in our case is a product description — essentially a set of data points or attributes we have extracted from a product page. These attributes include: title, brand, category, price, and other specs. Therefore, these are the attributes that help us cluster together similar products and match products. The quality of clustering — that is how accurate and how complete the clusters are — depends on how good the features are. In our case, most of the times the features are not good, and that is what makes clustering, and in turn product matching, a hard problem.

    Noisy Small Factually Weak (NSFW) Documents

    The documents that we deal with, the product descriptions, are not well formed and so not readily usable for product matching. We at PriceWeave characterize them endearignly as Noisy Weak and Factually Weak (NSFW) documents. Let us see some examples to understand these terms.

    Noisy

    • Spelling errors, non-standard and/or incomplete representations of product features.
    • Brands written as “UCB” and “WD” instead of “United Colors of Benetton” and “Western Digital”.
    • Model no.s might or might not be present. A camera’s model number written as one of the following variants: DSC-WX650 vs DSCWX650 vs DSC WX 650 vs WX 650.
    • Noisy/meaningless terms might be present (“brand new”, “manufacturer’s warranty”, “with purchase receipt”)

    Small

    • Not much description. A product simply written as “Apple iPhone” without any mention of its generation, or other features.
    • Not many distinguishable features. Example, “Samsung Galaxy Note vs Samsung Galaxy Note 2”, “Apple ipad 3 16 GB wifi+cellular vs Apple ipad mini 16 GB wifi-cellular”

    Factually Weak

    • Products represented with generic and subjective descriptions.
    • Colours and their combinations might be represented differently. Examples, “Puma Red Striped Bag”, “Adidas Black/Red/Blue Polo Tshirt”.

    In the absence of clean, sufficient, and specific product information, the quality of algorithmic matching suffers. Product matching include many knobs and switches to adjust the weights given to different product attributes. For example, we might include a rule that says, “if two products are identical, then they fall in the same price range.” While such rules work well generally, they vary widely from category to category and across geographies. Further, adding more and more specific rules will start throwing off the algorithms in unexpected ways rendering them less effective.

    In this post, we discussed the challenges posed by product matching that make it a hard problem to crack. In the next post, we will discuss how we address these challenges to make PriceWeave’s product matching robust.

    PriceWeave is an all-around Competitive Intelligence product for retailers, brands, and manufacturers. We’re built on top of huge amounts of products data to provide real-time actionable insights. PriceWeave’s offerings include: pricing intelligence, assortment intelligence, gaps in catalogs, and promotion analysis. Please visit PriceWeave to view all our offerings. If you’d like to try us out request for a demo.

    Originally published at blog.priceweave.com.