Amazon 2014 and the P5 compatibility layer¶
Older documentation called the main data path “P5 Amazon.” The current code has
two distinct implementations: model-specific Amazon 2014 adapters used by
benchmarks, and a legacy PyTorch Geometric/P5 pipeline in p5_amazon.py.
Current path: Amazon Reviews 2014 5-core¶
Expected raw files are:
dataset/amazon/raw/<split>/reviews_<Category>_5.json.gz
dataset/amazon/raw/<split>/meta_<Category>.json.gz
The current DATASET_CONFIGS entries are:
--split |
Review file | Metadata file |
|---|---|---|
beauty |
reviews_Beauty_5.json.gz |
meta_Beauty.json.gz |
sports |
reviews_Sports_and_Outdoors_5.json.gz |
meta_Sports_and_Outdoors.json.gz |
toys |
reviews_Toys_and_Games_5.json.gz |
meta_Toys_and_Games.json.gz |
clothing |
reviews_Clothing_Shoes_and_Jewelry_5.json.gz |
meta_Clothing_Shoes_and_Jewelry.json.gz |
home |
reviews_Home_and_Kitchen_5.json.gz |
meta_Home_and_Kitchen.json.gz |
The books and arts_crafts_and_sewing entries read compatible files produced
by the Amazon 2023 converter; they are not Stanford 2014 categories with the
same names.
Fields used by current adapters¶
| Source | Field | Use |
|---|---|---|
| Review | asin |
Item key |
| Review | reviewerID |
User key |
| Review | unixReviewTime |
Sequence ordering and temporal splits |
| Metadata | asin |
Alignment with the item mapping |
| Metadata | title, price, salesRank, brand, categories |
Text passed to AmazonItemDataset's encoder |
Ratings are not supervision targets for current next-item adapters. Items that lack usable metadata can disappear from the embedding table, so always check embedding rows against the sequence mapping.
Download and validate¶
The script serializes concurrent downloads, writes temporary files, reads to
gzip EOF to validate the stream, and records SHA-256 hashes. AmazonItemDataset
can also download missing files, but the explicit preparation script gives
clearer provenance on shared machines.
Text embeddings¶
AmazonItemDataset formats available metadata into a text record containing
title, price, sales rank, brand, and categories. SentenceTransformer encodes the
records in batches and writes parquet rows with:
| Column | Meaning |
|---|---|
ItemID |
Integer ID in the current mapping |
embedding |
Vector represented as a Python list |
The encoder basename is part of the cache name, for example
item_emb_sentence-t5-xl.parquet. TIGER, LCRec, COBRA, and RPG commonly use
Sentence-T5; OneRec's RQ-KMeans path uses Qwen3-Embedding-8B.
User sequences are adapter-specific¶
All adapters rebuild mappings from reviews and sort by timestamp, but their training samples differ:
- SASRec and HSTU keep one training sequence per user and supervise multiple shifted positions in the collate step;
- TIGER and LCRec commonly build sliding prefixes;
- OneRec creates natural-language/SID tasks;
- RPG uses 1-based IDs and right padding;
- NoteLLM creates adjacent item-to-item retrieval pairs.
Do not pass integer sequences serialized by one adapter to another adapter.
Legacy p5_amazon.py¶
The classes that actually exist are:
| Class | Purpose |
|---|---|
AmazonReviews |
PyG InMemoryDataset that builds a heterogeneous user-item graph and histories |
P5AmazonReviewsItemDataset |
Reads item embeddings from the PyG processed artifact |
P5AmazonReviewsSeqDataset |
Loads RQ-VAE and converts item histories into SeqData SIDs |
Names such as P5AmazonItemDataset and P5AmazonSequenceDataset do not exist in
this repository. The legacy module imports torch_geometric, has its own
processed artifacts and historical defaults, and is not bound by current Gin
configs. Use it only when reproducing an old checkpoint, with its data version
recorded explicitly.
Minimal examples¶
Item tokenizer input:
from genrec.data.amazon import AmazonItemDataset
items = AmazonItemDataset(
root="dataset/amazon",
split="beauty",
train_test_split="all",
encoder_model_name="/data/models/sentence-t5-xl",
)
print(len(items), items.dim)
SASRec evaluation input:
from genrec.data.amazon_sasrec import AmazonSASRecDataset
test = AmazonSASRecDataset(
root="dataset/amazon",
split="beauty",
train_test_split="test",
max_seq_len=50,
)
print(len(test), test.num_items, test[0])
Construction may read large raw files or encode metadata; these snippets assume that caches have already been prepared.
Data sources retain their own licenses. The repository's MIT license does not relicense Amazon data or pretrained models. See the Amazon Product Data page and the P5 paper.