The DataWeave Blog

Tag: Product Matching

Maximizing Competitive Match Rates: The Foundation of Effective Price Intelligence
Merchants make countless pricing decisions every day. Whether you’re a brand selling online, a traditional brick-and-mortar retailer, or another seller attempting to navigate the vast world of commerce, figuring out the most effective price intelligence strategy is essential. Having your plan in place will help you price your products in the sweet spot that enhances your price image and maximizes profits.

For the best chance of success, your overall pricing strategy must include competitive intelligence.

Many retailers focus their efforts on just collecting the data. But that’s only a portion of the puzzle. The real value lies in match accuracy and knowing exactly which competitor products to compare against. In this article, we will dive deeper into cutting-edge approaches that combine the traditional matching techniques you already leverage with AI to improve your match rates dramatically.

If you’re a pricing director, category manager, commercial leader, or anyone else who deals with pricing intelligence, this article will help you understand why competitive match rates matter and how you can improve yours.

Change your mindset from tactical to strategic and see the benefits in your bottom line.

The Match Rate Challenge

To the layman, tracking and comparing prices against the competition seems easy. Just match up two products and see which ones are the same! In reality, it’s much more challenging. There are thousands of products to discover, analyze, compare, and derive subjective comparisons from. Not only that, product catalogs across the market are constantly evolving and growing, so keeping up becomes a race of attrition with your competitors.

Let’s put it into focus. Imagine you’re trying to price a 12-pack of Coca-Cola. This is a well-known product that, hypothetically, should be easy to identify across the web. However, every retailer uses their own description in their listing. Some examples include:
- Retailer A lists it as “Coca-Cola 12 Fl. Oz 12 Pack”
- Retailer B shows “Coca Cola Classic Soda Pop Fridge Pack, 12 Fl. Oz Cans, 12-Pack”
- Retailer C has “Coca-Cola Soda – 12pk/12 fl oz Cans”
While a human can easily deduce that these are the same product, the automated system you probably have in place right now is most likely struggling. It cannot tell the difference between the retailers’ unique naming conventions, including brand name, description, bundle, unit count, special characters, or sizing.

This has real-world business impacts if your tools cannot accurately compare the price of a Coca-Cola 12-pack across the market.

Why Match Rates Matter

If your competitive match rates are poor, you aren’t seeing the whole picture and are either overcharging, undercharging, or reacting to market shifts too slowly.

Overcharging can result in lost sales, while undercharging may result in out-of-stock due to spikes in demand you haven’t accounted for. Both are recipes to lose out on potential revenue, disappoint customers, and drive business to your competitors.

What you need is a sophisticated matching capability that can handle the tracking of millions of competitive prices each week. It needs to be able to compare using hundreds of possible permutations, something that is impossible for pricing teams to do manually, especially at scale. With technology to make this connection, you aren’t missing out on essential competitive intelligence.

The Business Impact

Besides the bottom-line savings, accurately matching competitor products for pricing intelligence has other business impacts that can help your business. Adding technology to your workflow to improve match rates can help identify blind spots, improve decision quality, and improve operational efficiency.
- Pricing Blind Spots
  - Missing competitor prices on key products
  - Inability to detect competitive threats
  - Delayed response to market changes
- Decision Quality
  - Incomplete competitive coverage leads to suboptimal pricing
  - Risk of pricing decisions based on wrong product comparisons
- Operational Efficiency
  - Manual verification costs
  - Time spent reconciling mismatched products
  - Resources needed to maintain price position
Current Industry Challenges

As mentioned, the #1 reason businesses like yours probably aren’t already finding the most accurate matches is that not all sites carry comparable product codes. If every listing had a consistent product code, it would be very easy to match that code to your code base. In fact, most retailers currently only achieve 60-70% match rates using their traditional methods.

Different product naming conventions, constantly changing product catalogs, and regional product variations contribute to the industry challenges, not to mention the difficulty of finding brand equivalencies and private label comparisons across the competition. So, if you’re struggling, just know everyone else is as well. However, there is a significant opportunity to get ahead of your competition if you can improve your match rates with technology.

The Matching Hierarchy
- Direct Code Matching: There are a number of ways to start finding matches across the market. The base tier of the hierarchy of most accurate approaches is Direct Code matching. Most likely, your team already has a process in place that can compare UPC to UPC, for example. When no standard codes are listed, your team is left with a blind spot. This poses limitations in modern retail but is an essential first step to identifying the “low-hanging fruit” to start getting matches.
- Non-Code-Based Matching: The next level of the hierarchy is implementing non-code-based matching strategies. This is when there are no UPCs, DPCIs, ASINs, or other known codes that make it easy to do one-to-one comparisons. These tools can analyze complex metrics like direct size comparisons, unique product descriptions, and features to find more accurate matches. They can look deep into the listing to extract data points beyond a code, even going as far as analyzing images and video content to help find matches. Advanced technologies for competitive matching can help pricing teams by adding different comparison metrics to their arsenal beyond code-based.
- Private Label Conversions: Up until this level of the hierarchy, comparisons relied on direct comparisons. Finding identical codes and features and naming similarities is excellent for figuring out one-to-one comparisons, but when there is no similar product to compare with for pricing intelligence, things get more complicated. This is the third tier of the matching hierarchy. It’s the ability to find similar product matches for ‘like’ products. This can be used for private label conversions and to create meaningful comparisons without direct matches.
- Similar Size Mappings: This final rung on the matching hierarchy adds another layer of advanced calculations to the comparison capability. Often, retailers and merchants list a product with different sizing values. One may choose to bundle products, break apart packs to sell as single items or offer a special-sized product manufactured just for them.
While at the end of the day, the actual product is the same, when there are unusual size permutations, it can be hard to identify the similarities. Technology can help with value size relationships, package variation handling, size equalization, and unit normalization.

The AI Advantage

AI is the natural solution for efficiently executing competitive product matching at scale. DataWeave offers solutions for pricing teams to help them reach over 95% product match accuracy. The tools leverage the most modern Natural Language Processing models for ingesting and analyzing product descriptions. Image recognition capabilities apply methods such as object detection, background removal, and image quality enhancement to focus on an individual product’s key features to improve match accuracy.

Deep learning models have been trained on years of data to perform pattern recognition in product attributes and to learn from historical matches. All of these capabilities, and others, automate the attribute matching process, from code to image to feature description, to help pricing teams build the most accurate profile of products across the market for highly accurate pricing intelligence.

Implementation Strategy

We understand that moving away from manual product comparison methods can be challenging. Every organization is different, but some fundamental steps can be followed for success when leveling up your pricing teams’ workflow.
1. First, conduct a baseline assessment. Figure out where you are on the Matching hierarchy. Are you still only doing direct code-based comparisons? Has your team branched out to compare other non-code-based identifiers?
2. Next, establish clear match rate targets for yourself. If your current match rate is aligned with industry norms, strive to significantly improve it, aiming for a high alignment that supports maximizing the match rate. Break this down into achievable milestones across different stages of the implementation process.
3. Work with your vendor on quality control processes. It may be worth running your current process in tandem to be able to calculate the improvements in real time. With a veteran technology provider like DataWeave, you can rely on the most cutting-edge technology combined with human-in-the-loop checks and balances and a team of knowledgeable support personnel. Additionally, for teams wanting direct control, DataWeave’s Approve/Disapprove Module lets your team review and validate match recommendations before they go live, maintaining full oversight of the matching process.
4. The more data about your products it has, the better your match rates. DataWeave’s competitive intelligence tools also come with a built-in continuous improvement framework. Part of this is the human element that continually ensures high-quality matches, but another is the AI’s ‘learning’ capabilities. Every time the AI is exposed to a new scenario, it learns for the next time.
5. The final step, ensure cross-functional alignment is achieved. Every one from the C-Suite down should be able to access the synthesized information useful for their role without complex data to sift through. Customized dashboards and reports can help with this process.
Future-Proofing Match Rates

The world of retail is constantly evolving. If you don’t keep up, you’re going to be left behind. There are emerging retail channels, like the TikTok shop, and new product identification methods to leverage, like image comparisons. As more products enter the market along with new retailers, figuring out how to scale needs to be taken into consideration. It’s impossible to keep up with manual processes. Instead, think about maximizing your match rates every week and not letting them degrade over time. A combination of scale, timely action, and highly accurate match rates will help you price your products the most competitively.

Key Takeaways

Match rates are the foundation of pricing intelligence. You can evaluate how advanced your match rate strategy is based on the matching hierarchy. If you’re still early in your journey, you’re likely still relying on code-to-code matches. However, using a mix of AI and traditional methods, you can achieve a 95% accuracy rate on product matching, leading to overall higher competitive match rates. As a result, with continuous improvement, you will stay ahead of the competition even as the goalposts change and new variables are introduced to the competitive landscape.

Starting this process to add AI to your pricing strategy can be overwhelming. At DataWeave, we work with you to make the change easy. Talk to us today to know more.
February 5, 2025
Redefining Product Attribute Tagging With AI-Powered Retail Domain Language Models
In online retail, success hinges on more than just offering quality products at competitive prices. As eCommerce catalogs expand and consumer expectations soar, businesses face an increasingly complex challenge: How do you effectively organize, categorize, and present your vast product assortments in a way that enhances discoverability and drives sales?

Having complete and correct product catalog data is key. Effective product attribute tagging—a crucial yet frequently undervalued capability—helps in achieving this accuracy and completeness in product catalog data. While traditional methods of tagging product attributes have long struggled with issues of scalability, consistency, accuracy, and speed, a new thinking and fundamentally new ways of addressing these challenges are getting established. These follow from the revolution brought in Large Language Models but they fashion themselves as Small Language Models (SLM) or more precisely as Domain Specific Language Models. These can be potentially considered foundational models as they solve a wide variety of downstream tasks albeit within specific domains. They are a lot more efficient and do a much better job in those tasks compared to an LLM. .

Retail Domain Language Models (RLMs) have the potential to transform the eCommerce customer journey. As always, it’s never a binary choice. In fact, LLMs can be a great starting point since they provide an enhanced semantic understanding of the world at large: they can be used to mine structured information (e.g., product attributes and values) out of unstructured data (e.g., product descriptions), create baseline domain knowledge (e.g, manufacturer-brand mappings), augment information (e.g., image to prompt), and create first cut training datasets.

Powered by cutting-edge Generative AI and RLMs, next-generation attribute tagging solutions are transforming how online retailers manage their product catalog data, optimize their assortment, and deliver superior shopping experiences. As a new paradigm in search emerges – based more on intent and outcome, powered by natural language queries and GenAI based Search Agents – the capability to create complete catalog information and rich semantics becomes increasingly critical.

In this post, we’ll explore the crucial role of attribute tagging in eCommerce, delve into the limitations of conventional tagging methods, and unveil how DataWeave’s innovative AI-driven approach is helping businesses stay ahead in the competitive digital marketplace.

Why Product Attribute Tagging is Important in eCommerce

As the eCommerce landscape continues to evolve, the importance of attribute tagging will only grow, making it a pertinent focus for forward-thinking online retailers. By investing in robust attribute tagging systems, businesses can gain a competitive edge through improved product comparisons, more accurate matching, understanding intent, and enhanced customer search experiences.

Taxonomy Comparison and Assortment Gap Analysis

Products are categorized and organized differently on different retail websites. Comparing taxonomies helps in understanding focus categories and potential gaps in assortment breadth in relation to one’s competitors: missing product categories, sizes, variants or brands. It also gives insights into the navigation patterns and information architecture of one’s competitors. This can help in making search and navigation experience more efficient by fine tuning product descriptions to include more attributes and/or adding additional relevant filters to category listing pages.

For instance, check out the different Backpack categories on Amazon and Staples in the images below.

Or look at the nomenclature of categories for “Pens” on Amazon (left side of the image) and Staples (right side of the image) in the image below.

Assortment Depth Analysis

Another big challenge in eCommerce is the lack of standardization in retailer taxonomy. This inconsistency makes it difficult to compare the depth of product assortments across different platforms effectively. For instance, to categorize smartphones,
- Retailer A might organize it under “Electronics > Mobile Phones > Smartphones”
- Retailer B could use “Technology > Phones & Accessories > Cell Phones”
- Retailer C might opt for “Consumer Electronics > Smartphones & Tablets”
Inconsistent nomenclature and grouping create a significant hurdle for businesses trying to gain a competitive edge through assortment analysis. The challenge is exacerbated if you want to do an in-depth assortment depth analysis for one or more product attributes. For instance, look at the image below to get an idea of the several attribute variations for “Desks” on Amazon and Staples.

Custom categorization through attribute tagging is essential for conducting granular assortment comparisons, allowing companies to accurately assess their product offerings against those of competitors.

Enhancing Product Matching Capabilities

Accurate product matching across different websites is fundamental for competitive pricing intelligence, especially when matching similar and substitute products. Attribute tagging and extraction play a crucial role in this process by narrowing down potential matches more effectively, enabling matching for both exact and similar products, and tagging attributes such as brand, model, color, size, and technical specifications.

For instance, when choosing to match similar products in the Sofa category for 2-3 seater sofas from Wayfair and Overstock, tagging attributes like brand, color, size, and more is a must for accurate comparisons.

Taking a granular approach not only improves pricing strategies but also helps identify gaps in product offerings and opportunities for expansion.

Fix Content Gaps and improve Product Detail Page (PDP) Content

Attribute tagging plays a vital role in enhancing PDP content by ensuring adherence to brand integrity standards and content compliance guidelines across retail platforms. Tagging attributes allows for benchmarking against competitor content, identifying catalog gaps, and enriching listings with precise details.

This strategic tagging process can highlight missing or incomplete information, enabling targeted optimizations or even complete rewrites of PDP content to improve discoverability and drive conversions. With accurate attribute tagging, businesses can ensure each product page is fully optimized to capture consumer attention and meet retail standards.

Elevating the Search Experience

In today’s online retail marketplace, a superior search experience can be the difference between a sale and a lost customer. Through in-depth attribute tagging, vendors can enable more accurate filtering to improve search result relevance and facilitate easier product discovery for consumers.

By integrating rich product attributes extracted by AI into an in-house search platform, retailers can empower customers with refined and user-friendly search functionality. Enhanced search capabilities not only boost customer satisfaction but also increase the likelihood of conversions by helping shoppers find exactly what they’re looking for more quickly and with minimal effort.

Pitfalls of Conventional Product Tagging Methods

Traditional methods of attribute tagging, such as manual and rule-based systems, have been significantly enhanced by the advent of machine learning. While these approaches may have sufficed in the past, they are increasingly proving inadequate in the face of today’s dynamic and expansive online marketplaces.

Scalability

As eCommerce catalogs expand to include thousands or even millions of products, the limitations of machine learning and rule-based tagging become glaringly apparent. As new product categories emerge, these systems struggle to keep pace, often requiring extensive revisions to existing tagging structures.

Inconsistencies and Errors

Not only is reliance on an entirely human-driven tagging process expensive, but it also introduces a significant margin for error. While machine learning can automate the tagging process, it’s not without its limitations. Errors can occur, particularly when dealing with large and diverse product catalogs.

As inventories grow more complex to handle diverse product ranges, the likelihood of conflicting or erroneous rules increases. These inconsistencies can result in poor search functionality, inaccurate product matching, and ultimately, a frustrating experience for customers, drawing away the benefits of tagging in the first place.

Speed

When product information changes or new attributes need to be added, manually updating tags across a large catalog is a time-consuming process. Slow tagging processes make it difficult for businesses to quickly adapt to emerging market trends causing significant delays in listing new products, potentially missing crucial market opportunities.

How DataWeave’s Advanced AI Capabilities Revolutionize Product Tagging

Advanced solutions leveraging RLMs and Generative AI offer promising alternatives capable of overcoming these challenges and unlocking new levels of efficiency and accuracy in product tagging.

DataWeave automates product tagging to address many of the pitfalls of other conventional methods. We offer a powerful suite of capabilities that empower businesses to take their product tagging to new heights of accuracy and scalability with our unparalleled expertise.

Our sophisticated AI system brings an advanced level of intelligence to the tagging process.

RLMs for Enhanced Semantic Understanding

Semantic Understanding of Product Descriptions

RLMs analyze the meaning and context of product descriptions rather than relying on keyword matching.
Example: “Smartphone with a 6.5-inch display” and “Phone with a 6.5-inch screen” are semantically similar, though phrased differently.

Attribute Extraction

RLMs can identify important product attributes (e.g., brand, size, color, model) even from noisy or unstructured data.
Example: Extracting “Apple” as a brand, “128GB” as storage, and “Pink” as the color from a mixed description.

Identifying Implicit Relationships

RLMs find implicit relationships between products that traditional rule-based systems miss.
Example: Recognizing that “iPhone 12 Pro” and “Apple iPhone 12” are part of the same product family.

Synonym Recognition in Product Descriptions

Synonym Matching with Context

RLMs identify when different words or phrases describe the same product.
Examples: “Sneakers” = “Running Shoes”, “Memory” = “RAM” (in electronics)
Even subtle differences in wording, like “rose gold” vs “pink” are interpreted correctly.

Overcoming Brand-Specific Terminology

Some brands use their own terminologies (e.g., “Retina Display” for Apple).
RLMs can map proprietary terms to more generic ones (e.g., Retina Display = High-Resolution Display).

Dealing with Ambiguities

RLMs analyze surrounding text to resolve ambiguities in product descriptions.
Example: Resolving “charger” to mean a “phone charger” when matched with mobile phones.

Contextual Understanding for Improved Accuracy and Precision

By leveraging advanced natural language processing (NLP), DataWeave’s AI can process and understand the context of lengthy product descriptions and customer reviews, minimizing errors that often arise at human touch points. The solution processes and interprets information to extract key information to dramatically improve the overall accuracy of product tags.

It excels at grasping the subtle differences between similar products, sizes, colors and identifying and tagging minute differences between items, ensuring that each product is uniquely and accurately represented in a retailer’s catalog.

This has a major impact on product and similarity-based matching that can even help optimize similar and substitute product matching to enhance consumer search. At the same time, our AI can understand that the same term might have different meanings in various product categories, adapting its tagging approach based on the specific context of each item.

This deep comprehension ensures that even nuanced product attributes are accurately captured and tagged for easy discoverability by consumers.

Case Study: Niche Jewelry Attributes

DataWeave’s advanced AI can assist in labeling the subtle attributes of jewelry by analyzing product images and generating prompts to describe the image. In this example, our AI identifies the unique shapes and materials of each item in the prompts.

The RLM can then extract key attributes from the prompt to generate tags. This assists in accurate product matching for searches as well as enhanced product recommendations based on similarities.

This multi-model approach provides the flexibility to adapt as product catalogs expand while remaining consistent with tagging to yield more robust results for consumers.

Unparalleled Scalability

DataWeave can rapidly scale tagging for new categories. The solution is built to handle the demands of even the largest eCommerce catalogs enabling:
- Effortless management of extensive product catalogs: We can process and tag millions of products without compromising on speed or accuracy, allowing businesses to scale without limitations.
- Automated bulk tagging: New product lines or entire categories can be tagged automatically, significantly reducing the time and resources required for catalog expansion.
Normalizing Size and Color in Fashion

Style, color, and size are the core attributes in the fashion and apparel categories. Style attributes, which include design, appearance, and overall aesthetics, can be highly specific to individual product categories.

Our product matching engine can easily handle color and sizing complexity via our AI-driven approach combined with human verification. By leveraging advanced technology to identify and normalize identical and similar products from competitors, you can optimize your pricing strategy and product assortment to remain competitive. Using Generative AI in normalizing color and size in fashion is key to powering competitive pricing intelligence at DataWeave.

Continuous Adaptation and Learning

Our solution evolves with your business, improving continuously through feedback and customization for retailers’ specific product categories. The system can be fine-tuned to understand and apply specialized tagging for niche or industry-specific product categories. This ensures that tags remain relevant and accurate across diverse catalogs and as trends emerge.

The AI in our platform also continuously learns from user interactions and feedback, refining its tagging algorithms to improve accuracy over time.

Stay Ahead of the Competition With Accurate Attribute Tagging

In the current landscape, the ability to accurately and consistently tag product attributes is no longer a luxury—it’s essential for staying competitive. With advancements in Generative AI, companies like DataWeave are revolutionizing the way product tagging is handled, ensuring that every item in a retailer’s catalog is presented with precision and depth. As shoppers demand a more intuitive, seamless experience, next-generation tagging solutions are empowering businesses to meet these expectations head-on.

DataWeave’s innovative approach to attribute tagging is more than just a technical improvement; it’s a strategic advantage in an increasingly competitive market. By leveraging AI to scale and automate tagging processes, online retailers can keep pace with expansive product assortments, manage content more effectively, and adapt swiftly to changes in consumer behavior. In doing so, they can maintain a competitive edge.

To learn more, talk to us today!
November 14, 2024
Mastering Grocery Pricing Intelligence: A Strategic Approach for Modern Retailers
When egg prices surged 70% during the 2023 avian flu outbreak, grocery retailers faced a critical dilemma: maintain margins and risk losing customers, or absorb costs and watch profits evaporate. Similarly, rising olive oil and chocolate prices also had domino effects, cascading down from retailers to consumers. In each of these scenarios, those with sophisticated pricing intelligence systems adapted swiftly, finding the sweet spot between competitiveness and profitability. Others weren’t so fortunate.

This scenario continues to play out daily across thousands of products in the grocery sector. From breakfast cereals to fresh produce to bottled water, retailers must orchestrate pricing across a variety of categories – each with its own competitive dynamics, margin requirements, and price sensitivity patterns.

The Evolution of Grocery Pricing Intelligence

Imagine these scenarios in the grocery industry:
- Milk prices spike during a supply shortage.
- Your competitor drops egg prices by 20%.
- Fresh produce costs fluctuate with an unseasonable frost.
For grocery retailers, these aren’t occasional challenges—they’re Tuesday. Reacting to each pricing crisis as it comes isn’t just exhausting—it’s a recipe for shrinking margins and missed opportunities.

Think of it this way: If you’re constantly playing defense with your pricing strategy, you’re already two steps behind. Commoditized items like milk and eggs face intense price competition, while seasonal products and fresh produce demand constant attention. Simply matching competitor prices or adjusting for cost changes isn’t enough anymore. What’s needed is a proactive approach that anticipates market shifts before they happen and turns pricing challenges into competitive advantages. This is where price management comes in.

Price management has transformed from simple competitor checks into a strategic power play that can make or break a retailer’s market position. Weekly manual adjustments have given way to a long-term strategic view, driven by data analytics and market intelligence. Here are the basics of how price management in grocery retail works today.

Three Pillars of Grocery Price Management

1. Smart Data Collection: Building Your Foundation

The journey begins with comprehensive data collection and storage across your entire product ecosystem. This means:
- Complete Coverage Of All SKUs Across All Stores: Tracking prices for all SKUs across all stores, with particular attention to high-velocity items and volatile categories.
- Dynamic Monitoring: Tracking prices across different time frequencies as grocery prices are highly volatile for different categories. So daily tracking for volatile items like dairy and produce, and weekly for more stable categories may be needed.
- Competitive Intelligence: Gathering data not just on prices, but on promotions, pack sizes, and private label alternatives.
- Infrastructure to Support Large Volumes of Data: Partnering with external data and analytics providers to bridge the gap when retailers struggle with the scale of digital infrastructure these data sets require.
2. Intelligent Data Refinement: Making Sense of the Numbers

Raw data alone isn’t enough—it needs context and structure to become actionable intelligence. This is called Data Refinement—the process of establishing meaningful relationships within the data to facilitate the extraction of valuable insights. This refinement stage is closely tied to the data collection strategy, as the quality and depth of the insights derived depend on the accuracy and coverage of the collected data.

Data refinement includes several key processes:

Advanced Product Matching

Picture this: You’re tracking a competitor’s pricing on organic apples. Simple, right? Not quite. Yes, Universal Product Codes (UPCs) and Price Lookup Codes (PLUs) are present in Grocery to standardize product identification across different retailers—unlike the fashion industry’s endless style variations. Still, product matching isn’t as straightforward as scanning barcodes.

Here’s the catch: many retailer websites don’t display them. Then there’s the private label puzzle—your “Store’s Best” organic apples need to match against competitors’ house brands, each with their own unique UPC. Throw in different sizes (4 Apples vs. 1Kg of Apples), regional product names (fancy naming for plain old arugula), and international brand variations (like the name for Sprite in the USA and China), and you’ve got yourself a complex matching challenge that would make conventional pricing intelligence providers sweat.

Custom Product Relationships for Consistent Pricing and Competitive Positioning

Think like a shopper browsing the dairy aisle. You regularly buy your family’s favorite organic yogurt, the 24oz tub. But today, you notice the larger 32oz size is on sale – except the 24oz isn’t. As you stand there, confused, you wonder: Is the sale only for the bigger size? Did I miss a promotion? Should I buy the 32oz even though it’s more than I need?

For shoppers, this inconsistent pricing across product variations creates a frustrating experience. Establishing clear relationships between related items in your catalog is essential for maintaining consistent pricing and a coherent competitive strategy.

Start by linking products based on attributes like size, brand, and packaging. That way, when you adjust the price of the 32oz yogurt, the 24oz version automatically updates too – no more scrambling to ensure uniform pricing across your assortment. Similarly, products of the same brand but with flavor variations should be connected to keep pricing consistent.

Taking this one step further, mapping your competitors’ exact and similar products is crucial for comprehensive competitive intelligence. Distinguishing between premium and private label tiers, national brands, and regional players gives you a holistic view of the landscape. With this understanding, you can hone your pricing strategies to maintain a clear, compelling position across your entire category lineup.

Consistent pricing, whether across your own product variations or against competitors, provides clarity and accuracy in your overall competitive positioning. By establishing these logical connections, you avoid the customer confusion of seemingly random, inconsistent discounts – and ensure your pricing strategies work in harmony, not disarray.

The Role of AI and Data Sciences in Data Refinement

On the surface, linking products based on attributes like size, brand, and packaging seems like a no-brainer. But developing and maintaining the systems to accurately and automatically identify these connections? That’s a whole different animal.

Think about it – you’re not just dealing with text-based product titles and UPCs. There are images, videos, regional variations, private labels, and a whole host of other data types and industry nuances to account for.

Luckily, DataWeave is one of the few companies that’s truly cracked the code. Our multimodal AI models are trained to process all those diverse data formats – from granular product specs to zany regional produce names. And it’s not just about technology; we also harness the power of human intelligence.

See, in the grocery world, category managers are the real decision makers. They know their shelves inside and out and can spot those tricky connections in product matching, especially when they are not UPC-based. That’s why DataWeave built in a Human-in-the-Loop (HITL) process, where their AI systems continuously learn from expert feedback. It’s a feedback loop that allows our customers to pitch in and keep product relationships accurate, reliable, and always adapting to new market realities.

So while product mapping may seem straightforward on the surface, the reality is it takes some serious horsepower to do it right. Thankfully, DataWeave has both the technical chops and the grocery industry know-how to make it happen. Because when it comes to pricing intelligence, getting those product connections right is half the battle.

3. Strategic Implementation: Turning Insights into Action

The true value of pricing intelligence (PI) is realized through its strategic application. Although many view PI as a technical function, its strategic significance is increasing, particularly in the context of recent economic pressures like inflation. Here’s why:

Tactical vs Strategic Use of Data: From Standard Reporting to Competitive Analysis

Pricing intelligence has come a long way from the days of simply reacting to daily price changes. These days, it’s not just about firefighting—it’s about driving long-term strategy.

You can use pricing data to make quick, tactical adjustments, like matching a competitor’s sudden price drop on milk. Or, you can leverage that same data to predict market trends, optimize your product lineup, and shape your overall pricing strategy. Retailers who take that strategic view can get out ahead of the curve, anticipating shifts instead of just chasing them.

DataWeave supports both of these approaches. Our Standard Reporting tools give pricing managers the nitty-gritty details they need—current practices, historical patterns, and operational KPIs. It’s all the insights you’d expect for making those tactical, day-to-day tweaks.

In addition, DataWeave offers something more powerful: Competitive analysis. This is where pricing intelligence becomes a true strategic weapon. By providing a high-level view of market positioning, competitor moves, and untapped opportunities, competitive analysis empowers leadership to make proactive, big-picture decisions.

Armed with this broader perspective, retailers can start taking a more surgical approach. Maybe you need to adjust pricing zones to better meet customer demands. Or rethink your overall strategies to stay ahead of the competition, not just keep pace. It’s the difference between constantly putting out fires and systematically fortifying your entire pricing fortress.

Beyond Pricing: Comprehensive Data for Broader Insights

Pricing intelligence is just the tip of the iceberg. When you really start to refine and harness your data, the possibilities for grocery retailers expand far beyond simple price comparisons. Think about it – all that information you’re collecting on products, markets, and consumer behavior? That’s a goldmine waiting to be tapped. Sure, you can use it to keep a pulse on competitor pricing. But why stop there?

What if you could leverage that data to optimize your product assortment, making sure you’re stocking the right mix to meet customer demands? Or tap into predictive analytics to get a glimpse of future market shifts, so you can get out ahead of the curve? How about using it to streamline your supply chain, identify availability inefficiencies, and get products to shelves faster?

Sure, pricing intelligence will always be mission-critical. But when you couple it with these other data-driven insights, that’s when grocery retailing gets really interesting. It’s about evolving from a price-matching robot to a true strategic visionary, armed with the intelligence to take your business to new heights.

Looking Ahead: The Future of Grocery Pricing Intelligence

The grocery pricing landscape continues to evolve, driven by:
- Integration of AI and machine learning for predictive pricing
- Enhanced focus on omnichannel pricing consistency
- Growing importance of personalization in pricing strategies
Pricing intelligence isn’t just about having data—it’s about having the right data and knowing how to use it strategically. Success requires a comprehensive approach that combines robust data collection, sophisticated analysis, and strategic implementation.

By embracing modern pricing intelligence tools and strategies, grocery retailers can navigate market volatility, maintain competitive positioning, and drive sustainable growth. The key lies in building a pricing ecosystem that’s both sophisticated enough to handle complex data and flexible enough to adapt to changing market conditions.

Ready to transform your pricing strategy? Check out our grocery price tracker to get month-on-month updates on grocery prices in the real world. Contact us to learn how our advanced pricing intelligence solutions can help your business stay ahead in the competitive grocery market.
November 13, 2024
Using Siamese Networks to Power Accurate Product Matching in eCommerce
Retailers often compete on price to gain market share in high performance product categories. Brands too must ensure that their in-demand assortment is competitively priced across retailers. Commerce and digital shelf analytics solutions offer competitive pricing insights at both granular and SKU levels. Central to this intelligence gathering is a vital process: product matching.

Product matching or product mapping involves associating identical or similar products across diverse online platforms or marketplaces. The matching process leverages the capabilities of Artificial Intelligence (AI) to automatically create connections between various representations of identical or similar products. AI models create groups or clusters of products that are exactly the same or “similar” (based on some objectively defined similarity criteria) to solve different use cases for retailers and consumer brands.

Accurate product matching offers several key benefits for brands and retailers:
- Competitive Pricing: By identifying identical products across platforms, businesses can compare prices and adjust their strategies to remain competitive.
- Market Intelligence: Product matching enables brands to track their products’ performance across various retailers, providing valuable insights into market trends and consumer preferences.
- Assortment Planning: Retailers can analyze their product range against competitors, identifying gaps or opportunities in their offerings.
Why Product Matching is Incredibly Hard

But product matching stands out as one of the most demanding technical processes for commerce intelligence tools. Here’s why:

Data Complexity

Product information comes in various (multimodal) formats – text, images, and sometimes video. Each format presents its own set of challenges, from inconsistent naming conventions to varying image quality.

Data Variance

The considerable fluctuations in both data quality and quantity across diverse product categories, geographical regions, and websites introduce an additional layer of complexity to the product matching process.

Industry Specific Nuances

Industry specific nuances introduce unique challenges to product matching. Exact matching may make sense in certain verticals, such as matching part numbers in industrial equipment or identifying substitute products in pharmaceuticals. But for other industries, exactly matched products may not offer accurate comparisons.
- In the Fashion and Apparel industry, style-to-style matching, accommodating variants and distinguishing between core sizes and non-core sizes and age groups become essential for accurate results.
- In Home Improvement, the presence of unbranded products, private labels, and the preference for matching sets rather than individual items complicates the process.
- On the other hand, for grocery, product matching becomes intricate due to the distinction between item pricing and unit pricing. Managing the diverse landscape of different pack sizes, quantities, and packaging adds further layers of complexity.
Diverse Downstream Use Cases

The diverse downstream business applications give rise to various flavors of product matching tailored to meet specific needs and objectives.

In essence, while product matching is a critical component in eCommerce, its intricacies demand sophisticated solutions that address the above challenges.

To solve these challenges, at DataWeave, we’ve developed an advanced product matching system using Siamese Networks, a type of machine learning model particularly suited for comparison tasks.

Siamese Networks for Product Matching

Our methodology involves the use of ensemble deep learning architectures. In such cases, multiple AI models are trained and used simultaneously to ensure highly accurate matches. These models tackle NLP (natural language processing) and Computer Vision challenges specific to eCommerce. This technology helps us efficiently narrow down millions of product candidates to just 5-15 highly relevant matches.

The Tech Powering Siamese Networks

The key to our approach is creating what we call “embeddings” – think of these as unique digital fingerprints for each product. These embeddings are designed to capture the essence of a product in a way that makes similar products easy to identify, even when they look slightly different or have different names.

Our system learns to create these embeddings by looking at millions of product pairs. It learns to make the embeddings for similar products very close to each other while keeping the embeddings for different products far apart. This process, known as metric learning, allows our system to recognize product similarities without needing to put every product into a rigid category.

This approach is particularly powerful for eCommerce, where we often need to match products across different websites that might use different names or images for the same item. By focusing on the key features that make each product unique, our system can accurately match products even in challenging situations.

How Siamese Networks Work?

Imagine having a pair of identical twins who are experts at spotting similarities and differences. That’s essentially what a Siamese network is – a pair of identical AI systems working together to compare things.

How it works:
- Twin AI systems: Two identical AI systems look at two different products.
- Creating ‘fingerprints’ or ‘embedding’: Each system creates a unique ‘fingerprint’ of the product it’s looking at.
- Comparison: These ‘fingerprints’ are then compared to see how similar the products are.
Architecture

The architecture of a Siamese network typically consists of three main components: the shared network, the similarity metric, and the contrastive loss function.
- Shared Network: This is the ‘brain’ that creates the product ‘fingerprints’ or ‘embeddings.’ It is responsible for extracting meaningful feature representations from the input samples. This network is composed of layers of neural units that work together. Weight sharing between the twin networks ensures that the model learns to extract comparable features for similar inputs, providing a basis for comparison.
- Similarity Metric: After the shared network processes the inputs, a similarity metric is employed. This decides how alike two ‘fingerprints’ or ‘embeddings’ are. The selection of a similarity metric depends on the specific task and characteristics of the input data. Frequently used similarity metrics include the Euclidean distance, cosine similarity, or correlation coefficient, each chosen based on its suitability for the given context and desired outcomes.
- Loss Function: For training the Siamese network, a specialized loss function is used. This helps the system improve its comparison skills over time. It guides and trains the network to generate akin embeddings for similar inputs and disparate embeddings for dissimilar inputs.
  
  This is achieved by imposing penalties on the model when the distance or dissimilarity between similar pairs surpasses a designated threshold, or when the distance between dissimilar pairs falls below another predefined threshold. This training strategy ensures that the network becomes adept at discerning and encoding the desired level of similarity or dissimilarity in its learned embeddings.
How DataWeave Uses Siamese Networks for Product Matching

At DataWeave, we use Siamese Networks to match products across different retailer websites. Here’s how it works:

Pre-processing (Image Preparation)
- We collect product images from various websites.
- We clean these images up to make them easier for our AI to understand.
- We use techniques like cropping, flipping, and adjusting colors to help our AI recognize products even if the images are slightly different.
Training The AI
- We show our AI system millions of product images, teaching it to recognize similarities and differences.
- We use a special learning method called “Triplet Loss” to help our AI understand which products are the same and which are different.
- We’ve tested different AI structures to find the one that works best for product matching, including ResNet, EfficientNet, NFNet, and ViT.
Image Retrieval
- Once trained, our AI creates a unique “fingerprint” for each product image.
- We store these fingerprints in a smart database.
- When we need to find a match for a product, we:
  - Create a fingerprint for the new product.
  - Quickly search our database for the most similar fingerprints.
  - Return the top matching products.
Matches are then assigned a high or a low similarity score and segregated into “Exact Matches” or “Similar Matches.” For example, check out the image of this white shoe on the left. It has a low similarity score with the pink shoe (below) and so these SKUs are categorized as a “Similar Match.” Meanwhile, the shoe on the right is categorized as an “Exact Match.”

Similarly, in the following image of the dress for a young girl, the matched SKU has a high similarity score and so this pair is categorized as an “Exact Match.”

Siamese Networks play a pivotal role in DataWeave’s Product Matching Engine. Amid the millions of images and product descriptions online, our Siamese Networks act as an equalizing force, efficiently narrowing down millions of candidates to a curated selection of 10-15 potential matches.

In addition, these networks also find application in several other contexts at DataWeave. They are used to train our system to understand text-only data from product titles and joint multimodal content from product descriptions.

Leverage Our AI-Driven Product Matching To Get Insightful Data

In summary, accurate and efficient product matching is no longer a luxury – it’s a necessity. DataWeave’s advanced product matching solution provides brands and retailers with the tools they need to navigate this complex landscape, turning the challenge of product matching into a competitive advantage.

By leveraging cutting-edge technology and simplifying it for practical use, we empower businesses to make informed decisions, optimize their operations, and stay ahead in the ever-evolving eCommerce market. To learn more, reach out to us today!
June 26, 2024