indexer

package module
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 24, 2025 License: MPL-2.0 Imports: 44 Imported by: 0

README

FF-Indexer

Build Status Linter

A comprehensive NFT indexing system that tracks, processes, and provides access to NFT data across multiple blockchains including Ethereum, Tezos, and Bitmark.

Project Status

⚠️ This project is currently in active development and may contain bugs. While the core functionality is operational, some features may be unstable or incomplete. Please refer to the GitHub Issues for known problems and ongoing development tasks.

Overview

FF-Indexer is a microservices-based system designed to index NFT data from various blockchain networks and external APIs. It provides real-time event processing, comprehensive metadata indexing, image processing, and a GraphQL API for querying NFT information.

What It Solves

  • Multi-blockchain NFT Indexing: Supports Ethereum, Tezos, and Bitmark blockchains
  • Real-time Event Processing: Captures and processes NFT transfer, mint, and burn events as they happen
  • Cross-Marketplace Sales Data: Aggregates NFT sales data from multiple marketplaces including OpenSea, Blur, X2Y2, OBJKT, FxHash, etc.,
  • Comprehensive Metadata Management: Fetches and stores NFT metadata from IPFS and external sources
  • Image Processing: Downloads, processes, and generates thumbnails for NFT images
  • Provenance Tracking: Maintains complete ownership history and transaction records
  • Exchange Rate Tracking: Historical price data for various cryptocurrency pairs
  • GraphQL API: Flexible querying interface for NFT data

Architecture

The system consists of several microservices:

  • API Gateway: REST and GraphQL API endpoints
  • Event Processor: Processes blockchain events and coordinates workflows
  • Event Emitters: Monitor blockchain networks for NFT events (Ethereum & Tezos)
  • Workflow Runner: Executes background indexing workflows using Cadence
  • GRPC Gateway: Internal service communication
  • Image Indexer: Processes and stores NFT images
  • Provenance Indexer: Tracks ownership history and provenance

Quick Start

Prerequisites
  • Go 1.23+
  • MongoDB
  • PostgreSQL
  • Docker & Docker Compose (Recommended for quick setup)
  • Uber Cadence
Using Docker Compose
# Build and start all services in parallel
docker compose up --build -d

# Or using the script
make docker-build-ordered
Manual Setup
  1. Clone and build:
git clone git@github.com:feral-file/ff-indexer.git
cd ff-indexer
make build
  1. Configure services:
# Copy sample configs for each service
cp services/api-gateway/config.yaml.sample services/api-gateway/config.yaml
# Repeat for other services and update with your settings
  1. Run all services:
make run

Key Features

  • Multi-blockchain Support: Ethereum, Tezos, and Bitmark
  • Real-time Processing: Event-driven architecture with WebSocket connections
  • Scalable Architecture: Microservices with workflow orchestration
  • Image Processing: Automatic thumbnail generation and CDN integration
  • GraphQL API: Flexible data querying with strong typing
  • Comprehensive Testing: Unit tests and integration testing
  • Docker Support: Full containerization for easy deployment

Documentation

  • Development Guide - Detailed setup, architecture, and development information
  • System Diagrams - Architecture diagrams and component interactions
  • API Documentation - TBD

Configuration

Each service has its own configuration file. Sample configurations are provided in each service directory:

  • services/api-gateway/config.yaml.sample
  • services/event-processor/config.yaml.sample
  • services/workflow-runner/config.yaml.sample
  • And more...

Key configuration areas:

  • Database connections (MongoDB, PostgreSQL)
  • Blockchain RPC endpoints
  • External API keys (OpenSea, TZKT, etc.)
  • IPFS gateways
  • AWS services integration

API Usage

The system provides both REST and GraphQL endpoints:

REST API (v2 endpoints):

# Get account NFTs
GET /v2/nft?owner=<address>

# Query NFTs with filters
POST /v2/nft/query

# Get collections by creators
GET /v2/collections?creators=<creator-addresses>

GraphQL:

# Access GraphQL playground
GET /v2/graphiql

Contributing

  1. Fork the repository
  2. Create a feature branch
  3. Make your changes
  4. Add tests for new functionality
  5. Run the test suite: make test
  6. Run linter: make vet
  7. Submit a pull request
Code Style
  • Follow Go conventions and best practices
  • Use meaningful variable and function names
  • Add comments for complex logic
  • Write unit tests for new features

Testing

# Run all tests
make test

# Run linter
make vet

# Run specific service tests
go test ./services/api-gateway/...

Known Issues

Please refer to the GitHub Issues for current known problems, bug reports, and feature requests.

License

This project is licensed under the Mozilla Public License Version 2.0. See the LICENSE file for details.

Support

For issues and questions:

  1. Check existing GitHub Issues
  2. Create a new issue with detailed description
  3. Include relevant logs and configuration (without sensitive data)

Acknowledgments

Built with:

Documentation

Index

Constants

View Source
const (
	LivenetZeroAddress = "a3ezwdYVEVrHwszQrYzDTCAZwUD3yKtNsCq9YhEu97bPaGAKy1"
	TestnetZeroAddress = "dw9MQXcC5rJZb3QE1nz86PiQAheMP1dx9M3dr52tT8NNs14m33"
)
View Source
const (
	TezosNullAddress                 = "tz1Ke2h7sDdakHJQh8WX4Z372du1KChsksyU"
	TezosBurnAddress                 = "tz1burnburnburnburnburnburnburjAYjjX"
	TezosHicEtNuncMarketplaceAddress = "KT1HbQepzV1nVGg8QVznG7z4RcHseD5kwqBn"
	TezosTeiaMarketplaceAddress      = "KT1PHubm9HtyQEJ4BBpMTVomq6mhbfNZ9z5w"
	TezosOBJKTMarketplaceAddress     = "KT1FvqJwEDWb1Gwc55Jd1jjTHRVWbYKUUpyq"
	TezosOBJKTMarketplaceAddressV2   = "KT1WvzYHCNBvDSdwafTHv7nJ1dWmZ8GCYuuC"
	TezosOBJKTTreasuryProxyAddress   = "KT19kCpYFxrNPegMBYKKH44szaqv8offqBRz"
)

index excluded addresses

View Source
const (
	SourceFeralFile = "feralfile"
	SourceOpensea   = "opensea"
	SourceTZKT      = "tzkt"
)
View Source
const (
	ObjktCDNDisplayType           = "display"
	ObjktCDNArtifactType          = "artifact"
	ObjktCDNThumbnailType         = "thumbnail"
	ObjktCDNArtifactThumbnailType = "thumb288"
	ObjktCDNBasePath              = "/file/assets-003/"
	ObjktCDNHost                  = "assets.objkt.media"
)
View Source
const (
	DevelopmentEnvironment = "development"
	StagingEnvironment     = "staging"
	ProductionEnvironment  = "production"
)
View Source
const (
	// broken-image.svg
	DefaultDisplayURI          = "ipfs://QmX5rRzkZQfvEyaYc1Q78YZ83pFj3AgpFVSK8SmxUmZ85M"
	DefaultIPFSGateway         = "ipfs.nftstorage.link"
	FxhashGateway              = "gateway.fxhash.xyz"
	FxhashDevIPFSGateway       = "gateway.fxhash-dev2.xyz"
	FxhashWaitingToBeSignedCID = "QmbvEAn7FLMeYBDroYwBP8qWc3d3VVWbk19tTB83LCMB5S"
	FxhashOnchfsGateway        = "onchfs.fxhash2.xyz"
)
View Source
const (
	QueryPageSize       = 25
	UnsignedFxhashCID   = "QmYwSwa5hP4346GqD7hAjutwJSmeYTdiLQ7Wec2C7Cez1D"
	UnresolvedFxhashURL = "https://gateway.fxhash.xyz/ipfs//"
)
View Source
const (
	MediumUnknown  = "unknown"
	MediumVideo    = "video"
	MediumImage    = "image"
	MediumSoftware = "software"
	MediumOther    = "other"
)
View Source
const ENSContractAddress1 = "0x57f1887a8BF19b14fC0dF6Fd9B2acc9Af147eA85"

Ethereum contract addresses ignored contracts

View Source
const ENSContractAddress2 = "0xD4416b13d2b3a9aBae7AcD5D6C2BbDBE25686401"
View Source
const EthereumDeadAddress = "0x000000000000000000000000000000000000dEaD"
View Source
const EthereumZeroAddress = "0x0000000000000000000000000000000000000000"

index excluded addresses

View Source
const FXHASHContractAddressDev0_0 = "KT1NkZho1yRkDdQnN4Mz93sDYyY2pPrEHTNs"

development contract

View Source
const FXHASHContractAddressDev0_1 = "KT1TtVAyjh4Ahdm8sLZwFnL7tqoLf59XrK2h"
View Source
const FXHASHContractAddressFX0_0 = "KT1U6EHmNxJTkvaWJ4ThczG4FSDaHC21ssvi"

supported contract

View Source
const FXHASHContractAddressFX0_1 = "KT1KEa8z6vWXDJrVqtMrAeDVzsvxat3kHaCE"
View Source
const FXHASHContractAddressFX0_2 = "KT1AEVuykWeuuFX7QkEAMNtffzwhe1Z98hJS"
View Source
const FXHASHContractAddressFX1 = "KT1EfsNuqwLAWDd3o4pvfUx1CAh5GMdTrRvr"
View Source
const HicEtNuncContractAddress = "KT1RJ6PbjHpwc3M5rw5s2Nbmefwbuwbdxton"
View Source
const KALAMContractAddress = "KT1A5P4ejnLix13jtadsfV9GCnXLMNnab8UT"
View Source
const SeriesRegistryEventAssignSeriesSignature = "0x848e74caa538f5af05a1af3c186cca1e9d0c6f0c25995648face20eea1e930c7"
View Source
const SeriesRegistryEventDeleteSeriesSignature = "0xd1cb5d9dfefdfe2dda1152786a83292f2b48e6c71981358d0f8b9eb7dd92d6f4"
View Source
const SeriesRegistryEventOptInCollaborationSignature = "0xf1be82cf03ce2483decce6c97c3580b7298a5d3ba618b98bc41e7c208007f22f"
View Source
const SeriesRegistryEventOptOutSeriesSignature = "0xea7a51899c6e1e1b64a73aba8ff9ce974a129f879f83343326f6c09d42834620"
View Source
const SeriesRegistryEventRegisterSeriesSignature = "0x55d82c1e0fbf557aad06476685a2e64309e639e7b9763ffc3cffce16cb33f689"

Series Registry Contract

View Source
const SeriesRegistryEventUpdateArtistAddressSignature = "0x4908f8df298e29e82b08d2367b89cc9e0a041d1fa94df05fb14872975cd56597"
View Source
const SeriesRegistryEventUpdateSeriesSignature = "0x799805152d6bdc2303ac0dbac07df8ecbd5946b3f66426ab316f5f3670876096"
View Source
const TezDaoContractAddress = "KT1C9X9s5rpVJGxwVuHEVBLYEdAQ1Qw8QDjH"

Tezos contract addresses ignored contracts

View Source
const TezosDNSContractAddress = "KT1GBZmSxmnKJXGMdMLbugPfLyUPmuLSMwKS"
View Source
const (
	TezosOBJKTMarketplaceAddressTestnet = "KT1FvqJwEDWb1Gwc55Jd1jjTHRVWbYKUUpyq"
)

testent

View Source
const TransferEventSignature = "0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef"
View Source
const TransferSingleEventSignature = "0xc3d58168c5ae7397731d063d5bbf3d657854427343f4c083240f7aacaa2d0f62"
View Source
const VersumContractAddress = "KT1LjmAdYQCLBjwv4S2oFkEzyHVkomAf5MrW"

Variables

View Source
var (
	ErrTXNotFound            = fmt.Errorf("transaction is not found")
	ErrUnsupportedBlockchain = fmt.Errorf("unsupported blockchain")
)
View Source
var BlockchainAlias = map[string]string{
	utils.BitmarkBlockchain:  "bmk",
	utils.EthereumBlockchain: "eth",
	utils.TezosBlockchain:    "tez",
}
View Source
var ErrNoRecordUpdated = fmt.Errorf("no record updated")
View Source
var EthereumIndexExcludedOwners = map[string]struct{}{
	EthereumZeroAddress: {},
	EthereumDeadAddress: {},
}
View Source
var OBJKTSaleEntrypoints = []string{"fulfill_ask", "fulfill_offer"}

ObjktCDNTypes should be in order, make sure ObjktCDNThumbnailType stand behind ObjktCDNArtifactType

View Source
var SupportedCurrencyPairs = map[string]bool{
	"ETH-USD": true,
	"XTZ-USD": true,
}

Functions

func BuildQueryParams

func BuildQueryParams(params interface{}) string

func CheckCDNURLIsExist

func CheckCDNURLIsExist(url string) bool

CheckCDNURLIsExist check whether CDN URL exist or not

func DemoTokenPrefix

func DemoTokenPrefix(indexID string) string

func DownloadFile

func DownloadFile(c *http.Client, url string) (io.ReadCloser, string, error)

DownloadFile downloads a file from a given url and returns a file reader and its mime type

func ERC20Transfer

func ERC20Transfer(l types.Log) bool

func ERC721Transfer

func ERC721Transfer(l types.Log) bool

func ERC1155SingleTransfer

func ERC1155SingleTransfer(l types.Log) bool

func EthereumChecksumAddress

func EthereumChecksumAddress(address string) string
func GetCIDFromIPFSLink(link string) (string, error)

GetCIDFromIPFSLink searches and returns the CID for a give url

func GetETHBlockTime

func GetETHBlockTime(ctx context.Context, store cache.Store, rpcClient *ethclient.Client, hash common.Hash) (time.Time, error)

func GetMIMETypeByURL

func GetMIMETypeByURL(urlString string) string

GetMIMETypeByURL returns mimeType of a file based on the extension of the url

func HexSha1

func HexSha1(str string) string

func HexToDec

func HexToDec(hex string) string

convert HEX token to DEC format

func IsBurnAddress

func IsBurnAddress(address string, environment string) bool

IsBurnAddress returns true if the address is a burn address

func IsHTTPSURI

func IsHTTPSURI(uri string) bool

IsHTTPSURI returns true if the URI is an HTTPS URI

func IsIPFSLink(url string) bool

func IsIPFSURI

func IsIPFSURI(uri string) bool

IsIPFSURI returns true if the URI is an IPFS URI

func MakeCDNURIFromIPFSURI

func MakeCDNURIFromIPFSURI(assetURI, assetType, contract, tokenID string) (string, error)

MakeCDNURIFromIPFSURI create Objkt CDN uri from IPFS Uri(extract cid)

func NewStreamDownloader

func NewStreamDownloader(reader io.Reader, closeFunc func() error) io.ReadCloser

func NormalizeIndexIDs

func NormalizeIndexIDs(indexIDs []string, isConvertToDecimal bool) []string

NormalizeIndexIDs takes an array of token ids and return an array formatted token ids which includes formatting ethereum address and converting token id from hex to decimal if isConvertToDecimal is set to true. NOTE: There is no error return in this call.

func OpenseaTokenIDToHex

func OpenseaTokenIDToHex(tokenID string) (string, error)

func OptimizeFxHashIPFSURL

func OptimizeFxHashIPFSURL(url string) string

func OptimizedOpenseaImageURL

func OptimizedOpenseaImageURL(imageURL string) (string, error)

OptimizedOpenseaImageURL get the filename by inspect opensea's cdn link.

func ParseTokenIndexID

func ParseTokenIndexID(indexID string) (string, string, string, error)

ParseTokenIndexID return blockchainType, contractAddress and token id for a given indexID

func ReadFromURL

func ReadFromURL(ctx context.Context, url string, timeout time.Duration) ([]byte, error)

ReadFromURL reads the data from given URL within a timeout

func ResolveIPFSURI

func ResolveIPFSURI(gateway, uri string) string

ResolveIPFSURI converts an IPFS URI (ipfs://CID/path) to a HTTP URL using the specified gateway. If the URI is already an HTTPS URL, it returns it unchanged. For IPFS URIs, it constructs a URL in the format https://gateway/ipfs/CID/path to enable content retrieval through the gateway.

func SleepWithContext

func SleepWithContext(ctx context.Context, d time.Duration) bool

SleepWithContext will return whenever the slept time reached or context done It returns true if the context is done

func TokenIndexID

func TokenIndexID(blockchainType, contractAddress, id string) string

TokenIndexID returns blockchain-based unique token id. It is constructed by blockchain alias, token contract and token id.

func TransferEventLog

func TransferEventLog(l types.Log) bool

func TxURL

func TxURL(blockchain, environment, txID string) string

TxURL returns corresponded blockchain transaction URL

Types

type AbsentMIMETypeToken

type AbsentMIMETypeToken struct {
	IndexID    string `json:"indexID"`
	PreviewURL string `json:"previewURL"`
}

type Account

type Account struct {
	Account          string    `json:"account" bson:"account"`
	Blockchain       string    `json:"blockchain" bson:"blockchain"`
	LastUpdatedTime  time.Time `json:"lastUpdateTime" bson:"lastUpdateTime"`
	LastActivityTime time.Time `json:"lastActivityTime" bson:"lastActivityTime"`
}

type AccountIdentity

type AccountIdentity struct {
	AccountNumber   string    `json:"accountNumber" bson:"accountNumber"`
	Blockchain      string    `json:"blockchain" bson:"blockchain"`
	Name            string    `json:"name" bson:"name"`
	LastUpdatedTime time.Time `json:"-" bson:"lastUpdatedTime"`
}

type AccountToken

type AccountToken struct {
	BaseTokenInfo     `bson:",inline"` // the latest token info
	IndexID           string           `json:"indexID" bson:"indexID"`
	OwnerAccount      string           `json:"ownerAccount" bson:"ownerAccount"`
	Balance           int64            `json:"balance" bson:"balance"`
	LastActivityTime  time.Time        `json:"lastActivityTime" bson:"lastActivityTime"`
	LastRefreshedTime time.Time        `json:"lastRefreshedTime" bson:"lastRefreshedTime"`
	LastPendingTime   []time.Time      `json:"lastPendingTime" bson:"lastPendingTime,omitempty"`
	PendingTxs        []string         `json:"pendingTxs" bson:"pendingTxs,omitempty"`
}

type Artist

type Artist struct {
	ID   string `json:"id" structs:"id" bson:"id"`       // Artist blockchain address
	Name string `json:"name" structs:"name" bson:"name"` // <creator.user.username>,
	URL  string `json:"url" structs:"url" bson:"url"`    // <OpenseaAPI/creator.address>,
}

type AssetAttributes

type AssetAttributes struct {
	Configuration *AssetConfiguration `json:"configuration,omitempty" bson:"configuration,omitempty"`
}

type AssetConfiguration

type AssetConfiguration struct {
	Scaling         string  `json:"scaling" bson:"scaling"`                 // fit|fill
	BackgroundColor string  `json:"backgroundColor" bson:"backgroundColor"` // #000000
	MarginLeft      float64 `json:"marginLeft" bson:"marginLeft"`           // 0.1
	MarginRight     float64 `json:"marginRight" bson:"marginRight"`         // 0.1
	MarginTop       float64 `json:"marginTop" bson:"marginTop"`             // 0.1
	MarginBottom    float64 `json:"marginBottom" bson:"marginBottom"`       // 0.1
	AutoPlay        bool    `json:"autoPlay" bson:"autoPlay"`               // true|false
	Looping         bool    `json:"looping" bson:"looping"`                 // true|false
	Interactable    bool    `json:"interactable" bson:"interactable"`       // true|false
	Overridable     bool    `json:"overridable" bson:"overridable"`         // true|false
}

type AssetMetadata

type AssetMetadata struct {
	Project VersionedProjectMetadata `json:"project" bson:"project"`
}

type AssetMetadataDetail

type AssetMetadataDetail struct {
	AssetID string

	// marketplace information
	Source    string
	SourceURL string
	AssetURL  string

	Name        string
	Description string
	MIMEType    string
	Medium      Medium
	Minter      string

	ArtistID   string
	ArtistName string
	ArtistURL  string
	MaxEdition int64

	ThumbnailURI string
	DisplayURI   string
	PreviewURI   string

	IsBooleanAmount bool

	ArtworkMetadata map[string]interface{}
	Artists         []Artist
}

AssetMetadataDetail is a structure what contains the basic source information of the underlying asset

func NewAssetMetadataDetail

func NewAssetMetadataDetail(assetID string) *AssetMetadataDetail

func (*AssetMetadataDetail) FromFxhashObject

func (detail *AssetMetadataDetail) FromFxhashObject(o fxhash.ObjectDetail)

FromFxhashObject reads asset detail from an fxhash API object

func (*AssetMetadataDetail) FromObjkt

func (detail *AssetMetadataDetail) FromObjkt(objktToken objkt.Token)

FromObjkt reads asset detail from Objkt API object

func (*AssetMetadataDetail) FromOpenseaAsset

func (detail *AssetMetadataDetail) FromOpenseaAsset(a *opensea.DetailedAssetV2, source string)

func (*AssetMetadataDetail) FromTZIP21TokenMetadata

func (detail *AssetMetadataDetail) FromTZIP21TokenMetadata(md tzkt.TokenMetadata)

FromTZIP21TokenMetadata update TZKT token metadata to AssetMetadataDetail

func (*AssetMetadataDetail) FromTZKT

func (detail *AssetMetadataDetail) FromTZKT(t tzkt.Token)

FromTZKT reads asset detail from tzkt API object

func (*AssetMetadataDetail) ReplaceIPFSURIByObjktCDNURI

func (detail *AssetMetadataDetail) ReplaceIPFSURIByObjktCDNURI(assetType, assetURI, contract, tokenID string) string

ReplaceIPFSURIByObjktCDNURI return CDN uri if exist, if not this function will return ipfs link

func (*AssetMetadataDetail) SetMarketplace

func (detail *AssetMetadataDetail) SetMarketplace(profile MarketplaceProfile)

SetMarketplace sets marketplace property

func (*AssetMetadataDetail) SetMedium

func (detail *AssetMetadataDetail) SetMedium(m Medium)

func (*AssetMetadataDetail) UpdateMetadataFromObjkt

func (detail *AssetMetadataDetail) UpdateMetadataFromObjkt(token objkt.Token)

UpdateMetadataFromObjkt update Objkt metadata to AssetMetadataDetail

type AssetUpdateSet

type AssetUpdateSet struct {
	ID                 string                   `structs:"id,omitempty"`
	IndexID            string                   `structs:"indexID,omitempty"`
	Source             string                   `structs:"source,omitempty"`
	BlockchainMetadata interface{}              `structs:"blockchainMetadata,omitempty"`
	ProjectMetadata    VersionedProjectMetadata `structs:"projectMetadata,omitempty"`
	LastRefreshedTime  time.Time                `structs:"lastRefreshedTime"`
}

type AssetUpdates

type AssetUpdates struct {
	ID                 string          `json:"id"`
	Source             string          `json:"source"`
	ProjectMetadata    ProjectMetadata `json:"projectMetadata"`
	BlockchainMetadata interface{}     `json:"blockchainMetadata"`
	Tokens             []Token         `json:"tokens"`
}

AssetUpdates is the inputs payload of IndexAsset. It includes project metadata, blockchain metadata and tokens that is attached to it

type AssetV2

type AssetV2 struct {
	IndexID                   string           `json:"indexID" bson:"indexID"`
	ThumbnailID               string           `json:"thumbnailID" bson:"thumbnailID"`
	LastRefreshedTime         time.Time        `json:"lastRefreshedTime" bson:"lastRefreshedTime"`
	Attributes                *AssetAttributes `json:"attributes" bson:"attributes,omitempty"`
	Metadata                  AssetMetadata    `json:"metadata" bson:"metadata"`
	StaticPreviewURLLandscape *string          `json:"staticPreviewURLLandscape" bson:"staticPreviewURLLandscape"`
	StaticPreviewURLPortrait  *string          `json:"staticPreviewURLPortrait" bson:"staticPreviewURLPortrait"`
}

type BaseTokenInfo

type BaseTokenInfo struct {
	ID              string `json:"id" bson:"id"`
	Blockchain      string `json:"blockchain" bson:"blockchain"`
	Fungible        bool   `json:"fungible" bson:"fungible"`
	ContractType    string `json:"contractType" bson:"contractType"`
	ContractAddress string `json:"contractAddress,omitempty" bson:"contractAddress"`
}

type BlockCache

type BlockCache struct {
	BlockHash  string    `json:"blockHash" bson:"blockHash"`
	BlockTime  time.Time `json:"blockTime" bson:"blockTime"`
	Blockchain string    `json:"blockchain" bson:"blockchain"`
}

type BlockchainAddress

type BlockchainAddress string

BlockchainAddress is a type of blockchain addresses supported in indexer

func (BlockchainAddress) String

func (a BlockchainAddress) String() string

func (*BlockchainAddress) UnmarshalJSON

func (a *BlockchainAddress) UnmarshalJSON(data []byte) error

type Collection

type Collection struct {
	ID          string                 `json:"id" bson:"id"`
	ExternalID  string                 `json:"externalID" bson:"externalID"`
	Creators    []string               `json:"creators" bson:"creators"`
	Name        string                 `json:"name" bson:"name"`
	Description string                 `json:"description" bson:"description"`
	Items       int                    `json:"items" bson:"items"`
	ImageURL    string                 `json:"imageURL" bson:"imageURL"`
	Contracts   ContractAddresses      `json:"contracts" bson:"contracts"`
	Published   bool                   `json:"published" bson:"published"`
	Source      string                 `json:"source" bson:"source"`
	ExternalURL string                 `json:"externalURL" bson:"externalURL"`
	Metadata    map[string]interface{} `json:"metadata" bson:"metadata"`

	LastUpdatedTime time.Time `json:"lastUpdatedTime" bson:"lastUpdatedTime"`
	CreatedAt       time.Time `json:"createdAt" bson:"createdAt"`
}

type CollectionAsset

type CollectionAsset struct {
	CollectionID     string    `json:"collectionID" bson:"collectionID"`
	TokenIndexID     string    `json:"tokenIndexID" bson:"tokenIndexID"`
	Edition          int64     `json:"edition" bson:"edition"`
	LastActivityTime time.Time `json:"lastActivityTime" bson:"lastActivityTime"`

	RunID string `json:"-" bson:"runID"`
}

type CollectionUpdates

type CollectionUpdates struct {
	ID          string      `json:"id"`
	ExternalID  string      `json:"externalID"`
	Blockchain  string      `json:"blockchain"`
	Owner       string      `json:"owner"`
	Name        string      `json:"name"`
	Description string      `json:"description"`
	ImageURL    string      `json:"imageURL"`
	Contract    string      `json:"contract"`
	Metadata    interface{} `json:"metadata"`
	Published   bool        `json:"published"`
	Source      string      `json:"source"`
	SourceURL   string      `json:"source_url"`
}

CollectionUpdates is the inputs payload of IndexCollection

type ContractAddresses

type ContractAddresses struct {
	Ethereum EthereumContractAddresses `json:"ethereum"`
	Tezos    TezosContractAddresses    `json:"tezos"`
}

type ContractTokens

type ContractTokens map[string][]string

ContractTokens maps contract addresses to their token IDs

func (ContractTokens) ContractAddresses

func (ct ContractTokens) ContractAddresses() []string

ContractAddresses returns all contract addresses from ContractTokens

type Criteria

type Criteria struct {
	IndexID string `bson:"indexID"`
	Source  string `bson:"source"`
}

type DetailedToken

type DetailedToken struct {
	Token           `bson:",inline"`
	ThumbnailID     string                   `json:"thumbnailID"`
	IPFSPinned      bool                     `json:"ipfsPinned"`
	Attributes      *AssetAttributes         `json:"attributes,omitempty"`
	ProjectMetadata VersionedProjectMetadata `json:"projectMetadata" bson:"projectMetadata"`
}

DetailedToken is the summarized information of a token. It includes asset information that this token is linked to.

type DetailedTokenV2

type DetailedTokenV2 struct {
	Token      `bson:",inline"`
	IPFSPinned bool    `json:"ipfsPinned"`
	Asset      AssetV2 `json:"asset" bson:"asset"`
}

type EthereumContractAddresses

type EthereumContractAddresses struct {
	ERC721  []string `json:"erc721"`
	ERC1155 []string `json:"erc1155"`
}

type EthereumContracts

type EthereumContracts struct {
	ERC721  ContractTokens `json:"erc721"`
	ERC1155 ContractTokens `json:"erc1155"`
}

EthereumContracts holds Ethereum-specific contract types

func (EthereumContracts) ContractAddressMap

func (ec EthereumContracts) ContractAddressMap() EthereumContractAddresses

ContractAddressMap returns a mapped representation of Ethereum contracts

type ExchangeRate

type ExchangeRate struct {
	Timestamp    time.Time `json:"timestamp" bson:"timestamp"`
	Price        float64   `json:"price" bson:"price"`
	CurrencyPair string    `json:"currencyPair" bson:"currencyPair"`
}

type FilterParameter

type FilterParameter struct {
	Source         string
	IDs            []string
	BurnedIncluded bool
}

type GenericSalesTimeSeries

type GenericSalesTimeSeries struct {
	Timestamp string                 `json:"timestamp"`
	Metadata  map[string]interface{} `json:"metadata"`
	Values    map[string]string      `json:"values"`
	Shares    map[string]string      `json:"shares"`
}

type GrouppedTokenFeedback

type GrouppedTokenFeedback struct {
	IndexID   string              `bson:"_id" json:"indexID,omitempty"`
	MimeTypes []MimeTypeWithCount `bson:"mimeTypes" json:"mimeTypes"`
}

type HexString

type HexString string

func (HexString) MarshalJSON

func (s HexString) MarshalJSON() ([]byte, error)

func (*HexString) UnmarshalJSON

func (s *HexString) UnmarshalJSON(data []byte) error

type HistoricalExchangeRateFilter

type HistoricalExchangeRateFilter struct {
	CurrencyPair string
	Timestamp    time.Time
}

type IndexEngine

type IndexEngine struct {
	// contains filtered or unexported fields
}

func New

func New(
	environment string,
	ipfsGateways []string,
	minterGateways map[string]string,
	opensea *opensea.Client,
	tzkt *tzkt.TZKT,
	fxhash *fxhash.Client,
	objkt *objkt.Client,
	ethereum *ethclient.Client,
	cacheStore cache.Store,
	blockchainQueryClient *managedblockchainquery.ManagedBlockchainQuery,
) *IndexEngine

func (*IndexEngine) GetETHTransactionDetailsByPendingTx

func (e *IndexEngine) GetETHTransactionDetailsByPendingTx(ctx context.Context, client *ethclient.Client, txHash common.Hash, tokenID string) ([]TransactionDetails, error)

GetETHTransactionDetailsByPendingTx gets transaction details by a specific pendingTx

func (*IndexEngine) GetEditionNumberByName

func (e *IndexEngine) GetEditionNumberByName(name string) int64

GetEditionNumberByName inferences the edition number by the format #{numbers}

func (*IndexEngine) GetEthereumTxTimestamp

func (e *IndexEngine) GetEthereumTxTimestamp(ctx context.Context, txHashString string) (time.Time, error)

GetEthereumTxTimestamp returns the timestamp of an transaction if it exists

func (*IndexEngine) GetObjktToken

func (e *IndexEngine) GetObjktToken(ctx context.Context, contract, tokenID string) (objkt.Token, error)

func (*IndexEngine) GetTezosTokenByOwner

func (e *IndexEngine) GetTezosTokenByOwner(owner string, lastTime time.Time, offset int) ([]tzkt.OwnedToken, error)

func (*IndexEngine) GetTezosTxTimestamp

func (e *IndexEngine) GetTezosTxTimestamp(_ context.Context, txHashString string) (time.Time, error)

GetTezosTxTimestamp returns the timestamp of an transaction if it exists

func (*IndexEngine) GetTokenBalanceOfOwner

func (e *IndexEngine) GetTokenBalanceOfOwner(c context.Context, contract, tokenID, owner string) (int64, error)

GetTokenBalanceOfOwner returns the balance of a token for an owner

func (*IndexEngine) GetTransactionDetailsByPendingTx

func (e *IndexEngine) GetTransactionDetailsByPendingTx(pendingTx string) ([]tzkt.DetailedTransaction, error)

func (*IndexEngine) GetTxTimestamp

func (e *IndexEngine) GetTxTimestamp(ctx context.Context, blockchain, txHash string) (time.Time, error)

GetTxTimestamp returns transaction timestamp of a blockchain

func (*IndexEngine) GetTzktTransactionByContractsAndEntrypoint

func (e *IndexEngine) GetTzktTransactionByContractsAndEntrypoint(contracts []string, entrypoints []string, lastTime *time.Time, offset, limit int) ([]tzkt.Transaction, error)

GetTzktTransactionByID get tzkt transactions by transaction id

func (*IndexEngine) GetTzktTransactionByID

func (e *IndexEngine) GetTzktTransactionByID(id uint64) (tzkt.Transaction, error)

GetTzktTransactionByID get tzkt transaction by transaction id

func (*IndexEngine) GetTzktTransactionsByHash

func (e *IndexEngine) GetTzktTransactionsByHash(hash string) ([]tzkt.DetailedTransaction, error)

GetTzktTransactionByHash get tzkt transactions by transaction hash

func (*IndexEngine) IndexETHToken

func (e *IndexEngine) IndexETHToken(ctx context.Context, contract, tokenID string) (*AssetUpdates, error)

IndexETHToken indexes an Ethereum token with a specific contract and ID

func (*IndexEngine) IndexETHTokenByOwner

func (e *IndexEngine) IndexETHTokenByOwner(ctx context.Context, owner string, next string) ([]AssetUpdates, string, error)

IndexETHTokenByOwner indexes all tokens owned by a specific ethereum address

func (*IndexEngine) IndexETHTokenOwners

func (e *IndexEngine) IndexETHTokenOwners(contract, tokenID string) ([]OwnerBalance, error)

IndexETHTokenOwners indexes owners of a given token

func (*IndexEngine) IndexTezosToken

func (e *IndexEngine) IndexTezosToken(ctx context.Context, contract, tokenID string) (*AssetUpdates, error)

IndexTezosToken indexes a Tezos token with a specific contract and ID

func (*IndexEngine) IndexTezosTokenByOwner

func (e *IndexEngine) IndexTezosTokenByOwner(ctx context.Context, owner string, lastTime time.Time, offset int) ([]AssetUpdates, time.Time, error)

IndexTezosTokenByOwner indexes all tokens owned by a specific tezos address

func (*IndexEngine) IndexTezosTokenLastActivityTime

func (e *IndexEngine) IndexTezosTokenLastActivityTime(contract, tokenID string) (time.Time, error)

IndexTezosTokenLastActivityTime indexes the last activity timestamp of a given token

func (*IndexEngine) IndexTezosTokenOwners

func (e *IndexEngine) IndexTezosTokenOwners(contract, tokenID string) ([]OwnerBalance, error)

IndexTezosTokenOwners indexes owners of a given token

func (*IndexEngine) IndexTezosTokenProvenance

func (e *IndexEngine) IndexTezosTokenProvenance(contract, tokenID string) ([]Provenance, error)

IndexTezosTokenProvenance indexes provenance of a specific token

func (*IndexEngine) IndexToken

func (e *IndexEngine) IndexToken(c context.Context, contract, tokenID string) (*AssetUpdates, error)

IndexToken indexes tokens by detecting the blockchain of a contract. This function returns an `AssetUpdates“ object for using in `IndexAsset“ function

type MarketplaceProfile

type MarketplaceProfile struct {
	Source    string
	SourceURL string
	AssetURL  string
}

type Medium

type Medium string

type MimeTypeWithCount

type MimeTypeWithCount struct {
	MimeType string `bson:"mimeType" json:"mimeType"`
	Count    int    `bson:"count" json:"count"`
}

type MongodbIndexerStore

type MongodbIndexerStore struct {
	// contains filtered or unexported fields
}

func NewMongodbIndexerStore

func NewMongodbIndexerStore(ctx context.Context, mongodbURI, dbName, environment string) (*MongodbIndexerStore, error)

func (*MongodbIndexerStore) AggregateSaleRevenues

func (s *MongodbIndexerStore) AggregateSaleRevenues(ctx context.Context, filter SalesFilterParameter) (map[string]primitive.Decimal128, error)

AggregateSaleRevenues - get sale revenue group by currency belong to an address

func (*MongodbIndexerStore) CheckAddressOwnTokenByCriteria

func (s *MongodbIndexerStore) CheckAddressOwnTokenByCriteria(ctx context.Context, address string, criteria Criteria) (bool, error)

CheckAddressOwnTokenByCriteria returns true if address owns token

func (*MongodbIndexerStore) CheckAssetCreator

func (s *MongodbIndexerStore) CheckAssetCreator(
	ctx context.Context,
	IDs []string,
	creatorAddresses []string) (bool, error)

func (*MongodbIndexerStore) CountDetailedAccountTokensByOwner

func (s *MongodbIndexerStore) CountDetailedAccountTokensByOwner(ctx context.Context, owner string) (int64, error)

CountDetailedAccountTokensByOwner count the number of DetailedToken by owner

func (*MongodbIndexerStore) DeleteCollection

func (s *MongodbIndexerStore) DeleteCollection(ctx context.Context, collectionID string) error

DeleteCollection removes the collection and related assets

func (*MongodbIndexerStore) DeleteDemoTokens

func (s *MongodbIndexerStore) DeleteDemoTokens(ctx context.Context, owner string) error

DeleteDemoTokens deletes demo tokens which exclusively belong to an owner and updates demo tokens if they are related to other owners

func (*MongodbIndexerStore) DeleteDeprecatedCollectionAsset

func (s *MongodbIndexerStore) DeleteDeprecatedCollectionAsset(ctx context.Context, collectionID, runID string) error

DeleteDeprecatedCollectionAsset removes old tokens not belong the collection anymore

func (*MongodbIndexerStore) FilterBurnedIndexIDs

func (s *MongodbIndexerStore) FilterBurnedIndexIDs(ctx context.Context, indexIDs []string) ([]string, error)

FilterBurnedIndexIDs filter out burned tokens from provided list

func (*MongodbIndexerStore) FilterTokenIDsWithInconsistentProvenanceForOwner

func (s *MongodbIndexerStore) FilterTokenIDsWithInconsistentProvenanceForOwner(ctx context.Context, indexIDs []string, owner string) ([]string, error)

FilterTokenIDsWithInconsistentProvenanceForOwner returns a list of token ids where the latest token is not the given owner

func (*MongodbIndexerStore) GetAccount

func (s *MongodbIndexerStore) GetAccount(ctx context.Context, owner string) (Account, error)

GetAccount returns an account by a given address

func (*MongodbIndexerStore) GetCollectionByID

func (s *MongodbIndexerStore) GetCollectionByID(ctx context.Context, id string) (*Collection, error)

GetCollectionByID returns the collection by given id

func (*MongodbIndexerStore) GetCollectionLastUpdatedTime

func (s *MongodbIndexerStore) GetCollectionLastUpdatedTime(ctx context.Context, collectionID string) (time.Time, error)

GetCollectionLastUpdatedTime returns collection last refreshed time by collectionID

func (*MongodbIndexerStore) GetCollectionLastUpdatedTimeForCreator

func (s *MongodbIndexerStore) GetCollectionLastUpdatedTimeForCreator(ctx context.Context, creator string) (time.Time, error)

GetCollectionLastUpdateTimeeForOwner returns collection last refreshed time for an owner

func (*MongodbIndexerStore) GetCollectionsByCreators

func (s *MongodbIndexerStore) GetCollectionsByCreators(ctx context.Context, creators []string, offset, size int64) ([]Collection, error)

GetCollectionsByOwners returns list of collections for owners

func (*MongodbIndexerStore) GetDetailedAccountTokensByOwners

func (s *MongodbIndexerStore) GetDetailedAccountTokensByOwners(ctx context.Context, owner []string, filterParameter FilterParameter, lastUpdatedAt time.Time, sortBy string, offset, size int64) ([]DetailedTokenV2, error)

GetDetailedAccountTokensByOwners returns a list of DetailedToken by owner

func (*MongodbIndexerStore) GetDetailedToken

func (s *MongodbIndexerStore) GetDetailedToken(ctx context.Context, indexID string, burnedIncluded bool) (DetailedToken, error)

GetDetailedToken returns a token information based on indexID

func (*MongodbIndexerStore) GetDetailedTokens

func (s *MongodbIndexerStore) GetDetailedTokens(ctx context.Context, filterParameter FilterParameter, offset, size int64) ([]DetailedToken, error)

GetDetailedTokens returns a list of tokens information based on ids

func (*MongodbIndexerStore) GetDetailedTokensByCollectionID

func (s *MongodbIndexerStore) GetDetailedTokensByCollectionID(ctx context.Context, collectionID string, sortBy string, offset, size int64) ([]DetailedTokenV2, error)

GetDetailedTokensByCollectionID returns list of tokens by the collectionID

func (*MongodbIndexerStore) GetDetailedTokensByOwners

func (s *MongodbIndexerStore) GetDetailedTokensByOwners(ctx context.Context, owners []string, filterParameter FilterParameter, offset, size int64) ([]DetailedToken, error)

GetTokensByOwner returns a list of DetailedTokens which belong to an owner

func (*MongodbIndexerStore) GetDetailedTokensV2

func (s *MongodbIndexerStore) GetDetailedTokensV2(ctx context.Context, filterParameter FilterParameter, offset, size int64) ([]DetailedTokenV2, error)

GetDetailedTokensV2 returns a list of tokens information based on ids

func (*MongodbIndexerStore) GetExchangeRateLastTime

func (s *MongodbIndexerStore) GetExchangeRateLastTime(ctx context.Context) (time.Time, error)

func (*MongodbIndexerStore) GetHistoricalExchangeRate

func (s *MongodbIndexerStore) GetHistoricalExchangeRate(ctx context.Context, filter HistoricalExchangeRateFilter) (ExchangeRate, error)

func (*MongodbIndexerStore) GetIdentities

func (s *MongodbIndexerStore) GetIdentities(ctx context.Context, accountNumbers []string) (map[string]AccountIdentity, error)

GetIdentities returns a list of identities by a list of account numbers

func (*MongodbIndexerStore) GetIdentity

func (s *MongodbIndexerStore) GetIdentity(ctx context.Context, accountNumber string) (AccountIdentity, error)

GetIdentity returns an identity of an account

func (*MongodbIndexerStore) GetLatestActivityTimeByIndexIDs

func (s *MongodbIndexerStore) GetLatestActivityTimeByIndexIDs(ctx context.Context, indexIDs []string) (map[string]time.Time, error)

GetLatestActivityTimeByIndexIDs returns a list of latest value of lastActivityTime for account tokens groups by indexID

func (*MongodbIndexerStore) GetNullProvenanceTokensByIndexIDs

func (s *MongodbIndexerStore) GetNullProvenanceTokensByIndexIDs(ctx context.Context, indexIDs []string) ([]string, error)

GetNullProvenanceTokensByIndexIDs returns indexIDs that have null provenance

func (*MongodbIndexerStore) GetOwnedTokenIDsByOwner

func (s *MongodbIndexerStore) GetOwnedTokenIDsByOwner(ctx context.Context, owner string) ([]string, error)

GetOwnedTokenIDsByOwner returns a list of tokens which belongs to an owner

func (*MongodbIndexerStore) GetOwnerAccountsByIndexIDs

func (s *MongodbIndexerStore) GetOwnerAccountsByIndexIDs(ctx context.Context, indexIDs []string) ([]string, error)

GetOwnerAccountsByIndexIDs Get Owner Accounts By IndexIDs

func (*MongodbIndexerStore) GetOwnersByBlockchainContracts

func (s *MongodbIndexerStore) GetOwnersByBlockchainContracts(ctx context.Context, blockchainContracts map[string][]string) ([]string, error)

GetOwnersByBlockchainContracts returns owners by blockchain and contract

func (*MongodbIndexerStore) GetSaleTimeSeriesData

func (s *MongodbIndexerStore) GetSaleTimeSeriesData(ctx context.Context, filter SalesFilterParameter) ([]SaleTimeSeries, error)

func (*MongodbIndexerStore) GetTokenByIndexID

func (s *MongodbIndexerStore) GetTokenByIndexID(ctx context.Context, indexID string) (*Token, error)

func (*MongodbIndexerStore) GetTokensByIndexIDs

func (s *MongodbIndexerStore) GetTokensByIndexIDs(ctx context.Context, ids []string) ([]Token, error)

GetTokensByIndexIDs returns a list of tokens by a given list of index id

func (*MongodbIndexerStore) GetTokensByTextSearch

func (s *MongodbIndexerStore) GetTokensByTextSearch(ctx context.Context, searchText string, offset, size int64) ([]DetailedToken, error)

GetTokensByTextSearch returns a list of token those assets match have attributes that match the search text.

func (*MongodbIndexerStore) GetTotalBalanceOfOwnerAccounts

func (s *MongodbIndexerStore) GetTotalBalanceOfOwnerAccounts(ctx context.Context, addresses []string) (int, error)

GetTotalBalanceOfOwnerAccounts sum balance of ownerAccounts

func (*MongodbIndexerStore) Healthz

func (s *MongodbIndexerStore) Healthz(ctx context.Context) error

Healthz checks the db health status and returns errors if any

func (*MongodbIndexerStore) IndexAccount

func (s *MongodbIndexerStore) IndexAccount(ctx context.Context, account Account) error

IndexAccount indexes the account by inputs

func (*MongodbIndexerStore) IndexAccountTokens

func (s *MongodbIndexerStore) IndexAccountTokens(ctx context.Context, owner string, accountTokens []AccountToken) error

IndexAccountTokens indexes the account tokens by inputs

func (*MongodbIndexerStore) IndexAsset

func (s *MongodbIndexerStore) IndexAsset(ctx context.Context, id string, assetUpdates AssetUpdates) error

IndexAsset creates an asset and its corresponded tokens by inputs

func (*MongodbIndexerStore) IndexCollection

func (s *MongodbIndexerStore) IndexCollection(ctx context.Context, collection Collection) error

IndexCollection index new collection

func (*MongodbIndexerStore) IndexCollectionAsset

func (s *MongodbIndexerStore) IndexCollectionAsset(ctx context.Context, collectionID string, collectionAssets []CollectionAsset) error

IndexCollectionAsset index new collection tokens

func (*MongodbIndexerStore) IndexDemoTokens

func (s *MongodbIndexerStore) IndexDemoTokens(ctx context.Context, owner string, indexIDs []string) error

IndexDemoTokens copies existent tokens in the db but change the blockchain name to "demo" and modify owner Before indexing new demo tokens, all of the old ones of the same owner need to be deleted.

func (*MongodbIndexerStore) IndexIdentity

func (s *MongodbIndexerStore) IndexIdentity(ctx context.Context, identity AccountIdentity) error

IndexIdentity saves an identity into indexer store

func (*MongodbIndexerStore) MarkAccountTokenChanged

func (s *MongodbIndexerStore) MarkAccountTokenChanged(ctx context.Context, indexIDs []string) error

MarkAccountTokenChanged sets the lastRefreshedTime to now

func (*MongodbIndexerStore) PushProvenance

func (s *MongodbIndexerStore) PushProvenance(ctx context.Context, indexID string, lockedTime time.Time, provenance Provenance) error

PushProvenance push the latest provenance record for a token

func (*MongodbIndexerStore) ReplaceCollectionCreator

func (s *MongodbIndexerStore) ReplaceCollectionCreator(ctx context.Context, oldCreator, newCreator string) error

ReplaceCollectionCreator updates all occurrences of oldCreator to newCreator in the creators array of matching collections and updates the lastUpdatedTime.

func (*MongodbIndexerStore) SaleTimeSeriesDataExists

func (s *MongodbIndexerStore) SaleTimeSeriesDataExists(ctx context.Context, txID, blockchain string) (bool, error)

SaleTimeSeriesDataExists - check if a sale time series data exists for a transaction hash and blockchain

func (*MongodbIndexerStore) SwapToken

func (s *MongodbIndexerStore) SwapToken(ctx context.Context, swap SwapUpdate) (string, error)

SwapToken marks the original token to burned and creates a new token record which inherits original blockchain information

func (*MongodbIndexerStore) UpdateAccountTokenOwners

func (s *MongodbIndexerStore) UpdateAccountTokenOwners(ctx context.Context, indexID string, ownerBalances []OwnerBalance) error

UpdateAccountTokenOwners updates all account owners for a specific token

func (*MongodbIndexerStore) UpdateAssetsConfiguration

func (s *MongodbIndexerStore) UpdateAssetsConfiguration(
	ctx context.Context,
	IDs []string,
	configuration *AssetConfiguration) (int64, error)

func (*MongodbIndexerStore) UpdateCollectionCreators

func (s *MongodbIndexerStore) UpdateCollectionCreators(ctx context.Context, collectionID string, creators []string) error

UpdateCollectionCreators sync the creators to the creators array of the collection identified by collectionID, if not already present, and updates lastUpdatedTime.

func (*MongodbIndexerStore) UpdateOwner

func (s *MongodbIndexerStore) UpdateOwner(ctx context.Context, indexID string, owner string, updatedAt time.Time) error

UpdateOwner updates owner for a specific non-fungible token

func (*MongodbIndexerStore) UpdateOwnerForFungibleToken

func (s *MongodbIndexerStore) UpdateOwnerForFungibleToken(ctx context.Context, indexID string, lockedTime time.Time, to string, total int64) error

UpdateOwnerForFungibleToken adds a new owner to a fungible token

func (*MongodbIndexerStore) UpdateTokenOwners

func (s *MongodbIndexerStore) UpdateTokenOwners(ctx context.Context, indexID string, lastActivityTime time.Time, ownerBalances []OwnerBalance) error

UpdateTokenOwners updates owners for a specific token

func (*MongodbIndexerStore) UpdateTokenProvenance

func (s *MongodbIndexerStore) UpdateTokenProvenance(ctx context.Context, indexID string, provenances []Provenance) error

UpdateTokenProvenance updates provenance for a specific token

func (*MongodbIndexerStore) WriteHistoricalExchangeRate

func (s *MongodbIndexerStore) WriteHistoricalExchangeRate(ctx context.Context, records []coinbase.HistoricalExchangeRate) error

func (*MongodbIndexerStore) WriteTimeSeriesData

func (s *MongodbIndexerStore) WriteTimeSeriesData(
	ctx context.Context,
	records []GenericSalesTimeSeries,
) error

WriteTimeSeriesData - validate and store a time series record

type OwnerBalance

type OwnerBalance struct {
	Address  string    `json:"address"`
	Balance  int64     `json:"balance,string"`
	LastTime time.Time `json:"lastTime"`
}

type ProjectMetadata

type ProjectMetadata struct {
	// Common attributes
	ArtistID            string   `json:"artistID" structs:"artistID" bson:"artistID"`       // Artist blockchain address
	ArtistName          string   `json:"artistName" structs:"artistName" bson:"artistName"` // <creator.user.username>,
	ArtistURL           string   `json:"artistURL" structs:"artistURL" bson:"artistURL"`    // <OpenseaAPI/creator.address>,
	Artists             []Artist `json:"artists" structs:"artists" bson:"artists"`
	AssetID             string   `json:"assetID" structs:"assetID" bson:"assetID"`                                     // <asset_contract.address>,
	Title               string   `json:"title" structs:"title" bson:"title"`                                           // <name>,
	Description         string   `json:"description" structs:"description" bson:"description"`                         // <description>,
	MIMEType            string   `json:"mimeType" structs:"mimeType" bson:"mimeType"`                                  // <mime_type from file extension or metadata>,
	Medium              Medium   `json:"medium" structs:"medium" bson:"medium"`                                        // <"image" if image_url is present; "other" if animation_url is present> ,
	MaxEdition          int64    `json:"maxEdition" structs:"maxEdition" bson:"maxEdition"`                            // 0,
	BaseCurrency        string   `json:"baseCurrency,omitempty" structs:"baseCurrency" bson:"baseCurrency"`            // null,
	BasePrice           float64  `json:"basePrice,omitempty" structs:"basePrice" bson:"basePrice"`                     // null,
	Source              string   `json:"source" structs:"source" bson:"source"`                                        // <Opeasea/Artblock>,
	SourceURL           string   `json:"sourceURL" structs:"sourceURL" bson:"sourceURL"`                               // <linktoSourceWebsite>,
	PreviewURL          string   `json:"previewURL" structs:"previewURL" bson:"previewURL"`                            // <image_url or animation_url>,
	ThumbnailURL        string   `json:"thumbnailURL" structs:"thumbnailURL" bson:"thumbnailURL"`                      // <image_thumbnail_url>,
	GalleryThumbnailURL string   `json:"galleryThumbnailURL" structs:"galleryThumbnailURL" bson:"galleryThumbnailURL"` // <image_thumbnail_url>,
	AssetData           string   `json:"assetData" structs:"assetData" bson:"assetData"`                               // null,
	AssetURL            string   `json:"assetURL" structs:"assetURL" bson:"assetURL"`                                  // <permalink>

	// autonomy customized attributes
	Attributes *AssetAttributes `json:"attributes,omitempty" structs:"attributes,omitempty" bson:"attributes,omitempty"`

	// artwork metadata from source. currently on for Feral File
	ArtworkMetadata map[string]interface{} `json:"artworkMetadata" structs:"artworkMetadata" bson:"artworkMetadata"`

	// Operation attributes
	LastUpdatedAt time.Time `json:"lastUpdatedAt" structs:"lastUpdatedAt" bson:"lastUpdatedAt"`

	// Feral File attributes
	InitialSaleModel string `json:"initialSaleModel" structs:"initialSaleModel" bson:"initialSaleModel"` // airdrop|fix-price|highest-bid-auction|group-auction

	// Deprecated attributes
	OriginalFileURL string `json:"originalFileURL" structs:"-" bson:"-"`
}

type Provenance

type Provenance struct {
	// this field is only for ownership validating
	FormerOwner *string `json:"formerOwner,omitempty" bson:"-"`

	Type        string    `json:"type" bson:"type"`
	Owner       string    `json:"owner" bson:"owner"`
	Blockchain  string    `json:"blockchain" bson:"blockchain"`
	BlockNumber *uint64   `json:"blockNumber,omitempty" bson:"blockNumber,omitempty"` // TODO: make it non-nullable, just temporarily support the compatibility
	Timestamp   time.Time `json:"timestamp" bson:"timestamp"`
	TxID        string    `json:"txid" bson:"txid"`
	TxURL       string    `json:"txURL" bson:"txURL"`
}

type SaleTimeSeries

type SaleTimeSeries struct {
	Timestamp     time.Time              `json:"timestamp" bson:"timestamp"`
	Metadata      map[string]interface{} `json:"metadata" bson:"metadata"`
	Shares        map[string]interface{} `json:"shares" bson:"shares"`
	NetValue      primitive.Decimal128   `json:"netRevenue" bson:"netRevenue"`
	PaymentAmount primitive.Decimal128   `json:"paymentAmount" bson:"paymentAmount"`
	PlatformFee   primitive.Decimal128   `json:"platformFee" bson:"platformFee"`
	USDQuote      primitive.Decimal128   `json:"exchangeRate" bson:"exchangeRate"`
	Price         primitive.Decimal128   `json:"price" bson:"price"`
}

type SalesFilterParameter

type SalesFilterParameter struct {
	Addresses   []string
	Marketplace string
	From        *time.Time
	To          *time.Time
	Limit       int64
	Offset      int64
	SortASC     bool
}

type SeriesMetadata

type SeriesMetadata struct {
	Name            string                 `json:"name"`
	Description     string                 `json:"description"`
	Image           string                 `json:"image"`
	ExternalURL     string                 `json:"external_url"`
	Metadata        map[string]interface{} `json:"metadata"`
	MetadataVersion string                 `json:"metadata_version"`
}

type Store

type Store interface {
	Healthz(ctx context.Context) error
	IndexAsset(ctx context.Context, id string, assetUpdates AssetUpdates) error
	SwapToken(ctx context.Context, swapUpdate SwapUpdate) (string, error)
	UpdateOwner(ctx context.Context, indexID, owner string, updatedAt time.Time) error
	UpdateTokenProvenance(ctx context.Context, indexID string, provenances []Provenance) error
	UpdateTokenOwners(ctx context.Context, indexID string, lastActivityTime time.Time, ownerBalances []OwnerBalance) error
	PushProvenance(ctx context.Context, indexID string, lockedTime time.Time, provenance Provenance) error
	FilterTokenIDsWithInconsistentProvenanceForOwner(ctx context.Context, indexIDs []string, owner string) ([]string, error)
	GetTokensByIndexIDs(ctx context.Context, indexIDs []string) ([]Token, error)
	GetTokenByIndexID(ctx context.Context, indexID string) (*Token, error)
	GetOwnedTokenIDsByOwner(ctx context.Context, owner string) ([]string, error)
	GetDetailedTokens(ctx context.Context, filterParameter FilterParameter, offset, size int64) ([]DetailedToken, error)
	GetDetailedTokensByOwners(ctx context.Context, owner []string, filterParameter FilterParameter, offset, size int64) ([]DetailedToken, error)
	GetTokensByTextSearch(ctx context.Context, searchText string, offset, size int64) ([]DetailedToken, error)
	GetIdentity(ctx context.Context, accountNumber string) (AccountIdentity, error)
	GetIdentities(ctx context.Context, accountNumbers []string) (map[string]AccountIdentity, error)
	IndexIdentity(ctx context.Context, identity AccountIdentity) error
	IndexAccount(ctx context.Context, account Account) error
	IndexAccountTokens(ctx context.Context, owner string, accountTokens []AccountToken) error
	GetAccount(ctx context.Context, owner string) (Account, error)
	UpdateAccountTokenOwners(ctx context.Context, indexID string, tokenBalances []OwnerBalance) error
	IndexDemoTokens(ctx context.Context, owner string, indexIDs []string) error
	DeleteDemoTokens(ctx context.Context, owner string) error
	UpdateOwnerForFungibleToken(ctx context.Context, indexID string, lockedTime time.Time, to string, total int64) error
	GetLatestActivityTimeByIndexIDs(ctx context.Context, indexIDs []string) (map[string]time.Time, error)
	MarkAccountTokenChanged(ctx context.Context, indexIDs []string) error
	GetDetailedTokensV2(ctx context.Context, filterParameter FilterParameter, offset, size int64) ([]DetailedTokenV2, error)
	GetDetailedAccountTokensByOwners(ctx context.Context, owner []string, filterParameter FilterParameter, lastUpdatedAt time.Time, sortBy string, offset, size int64) ([]DetailedTokenV2, error)
	CountDetailedAccountTokensByOwner(ctx context.Context, owner string) (int64, error)
	GetDetailedToken(ctx context.Context, indexID string, burnedIncluded bool) (DetailedToken, error)
	GetTotalBalanceOfOwnerAccounts(ctx context.Context, addresses []string) (int, error)
	GetNullProvenanceTokensByIndexIDs(ctx context.Context, indexIDs []string) ([]string, error)
	GetOwnerAccountsByIndexIDs(ctx context.Context, indexIDs []string) ([]string, error)
	CheckAddressOwnTokenByCriteria(ctx context.Context, address string, criteria Criteria) (bool, error)
	GetOwnersByBlockchainContracts(context.Context, map[string][]string) ([]string, error)
	IndexCollection(ctx context.Context, collection Collection) error
	IndexCollectionAsset(ctx context.Context, collectionID string, collectionAssets []CollectionAsset) error
	DeleteCollection(ctx context.Context, collectionID string) error
	ReplaceCollectionCreator(ctx context.Context, oldCreator, newCreator string) error
	UpdateCollectionCreators(ctx context.Context, collectionID string, creators []string) error
	DeleteDeprecatedCollectionAsset(ctx context.Context, collectionID, runID string) error
	GetCollectionLastUpdatedTimeForCreator(ctx context.Context, creator string) (time.Time, error)
	GetCollectionLastUpdatedTime(ctx context.Context, collectionID string) (time.Time, error)
	GetCollectionByID(ctx context.Context, id string) (*Collection, error)
	GetCollectionsByCreators(ctx context.Context, creators []string, offset, size int64) ([]Collection, error)
	GetDetailedTokensByCollectionID(ctx context.Context, collectionID string, sortBy string, offset, size int64) ([]DetailedTokenV2, error)
	FilterBurnedIndexIDs(ctx context.Context, indexIDs []string) ([]string, error)
	WriteTimeSeriesData(
		ctx context.Context,
		records []GenericSalesTimeSeries,
	) error
	SaleTimeSeriesDataExists(ctx context.Context, txID, blockchain string) (bool, error)
	GetSaleTimeSeriesData(ctx context.Context, filter SalesFilterParameter) ([]SaleTimeSeries, error)
	AggregateSaleRevenues(ctx context.Context, filter SalesFilterParameter) (map[string]primitive.Decimal128, error)
	WriteHistoricalExchangeRate(ctx context.Context, exchangeRate []coinbase.HistoricalExchangeRate) error
	GetHistoricalExchangeRate(ctx context.Context, filter HistoricalExchangeRateFilter) (ExchangeRate, error)
	GetExchangeRateLastTime(ctx context.Context) (time.Time, error)
	UpdateAssetsConfiguration(ctx context.Context, IDs []string, configuration *AssetConfiguration) (int64, error)
	CheckAssetCreator(ctx context.Context, IDs []string, creatorAddresses []string) (bool, error)
}

type StreamDownloader

type StreamDownloader struct {
	io.Reader
	// contains filtered or unexported fields
}

func (StreamDownloader) Close

func (d StreamDownloader) Close() error

type SwapUpdate

type SwapUpdate struct {
	OriginalTokenID         string          `json:"originalTokenID"`
	OriginalBlockchain      string          `json:"originalBlockchain"`
	OriginalContractAddress string          `json:"originalContractAddress"`
	NewTokenID              string          `json:"newTokenID"`
	NewBlockchain           string          `json:"newBlockchain"`
	NewContractAddress      string          `json:"newContractAddress"`
	NewContractType         string          `json:"newContractType"`
	ProjectMetadata         ProjectMetadata `json:"projectMetadata"`
	BlockchainMetadata      interface{}     `json:"blockchainMetadata"`
}

SwapUpdate is the inputs payload for swap a token

type TezosContractAddresses

type TezosContractAddresses struct {
	FA2 []string `json:"fa2"`
}

type TezosContracts

type TezosContracts struct {
	FA2 ContractTokens `json:"fa2"`
}

TezosContracts holds Tezos-specific contract types

func (TezosContracts) ContractAddressMap

func (tc TezosContracts) ContractAddressMap() TezosContractAddresses

ContractAddressMap returns a mapped representation of Tezos contracts

type TezosTokenMetadata

type TezosTokenMetadata struct {
	TokenID   string               `json:"token_id"`
	TokenInfo map[string]HexString `json:"token_info"`
}

type Token

type Token struct {
	BaseTokenInfo   `bson:",inline"` // the latest token info
	Edition         int64            `json:"edition" bson:"edition"`
	EditionName     string           `json:"editionName" bson:"editionName"`
	MintedAt        time.Time        `json:"mintedAt" bson:"mintedAt"`
	Balance         int64            `json:"balance" bson:"-"` // a temporarily state of balance for a specific owner
	Owner           string           `json:"owner" bson:"owner"`
	Owners          map[string]int64 `json:"owners" bson:"owners"`
	OwnersArray     []string         `json:"-" bson:"ownersArray"`
	AssetID         string           `json:"-" bson:"assetID"`
	OriginTokenInfo []BaseTokenInfo  `json:"originTokenInfo" bson:"originTokenInfo"`
	IsDemo          bool             `json:"-" bson:"isDemo"`

	IndexID           string       `json:"indexID" bson:"indexID"`
	Source            string       `json:"source" bson:"source"`
	Swapped           bool         `json:"swapped" bson:"swapped"`
	SwappedFrom       *string      `json:"-" bson:"swappedFrom,omitempty"`
	SwappedTo         *string      `json:"-" bson:"swappedTo,omitempty"`
	Burned            bool         `json:"burned" bson:"burned"`
	Provenances       []Provenance `json:"provenance" bson:"provenance"`
	LastActivityTime  time.Time    `json:"lastActivityTime" bson:"lastActivityTime"`
	LastRefreshedTime time.Time    `json:"lastRefreshedTime" bson:"lastRefreshedTime"`
}

Token is a structure for token information

type TokenDetail

type TokenDetail struct {
	Edition  int64
	Fungible bool
	MintedAt time.Time
}

TokenDetail saves token specific detail from different sources

type TokenFeedback

type TokenFeedback struct {
	IndexID         string    `json:"indexID" bson:"indexID"`
	MimeType        string    `json:"mimeType" bson:"mimeType"`
	LastUpdatedTime time.Time `json:"lastUpdatedTime" bson:"lastUpdatedTime"`
	DID             string    `json:"did" bson:"did"`
}

type TokenFeedbackUpdate

type TokenFeedbackUpdate struct {
	IndexID  string `json:"indexID"`
	MimeType string `json:"mimeType"`
}

type TokenRegistry

type TokenRegistry struct {
	Ethereum EthereumContracts `json:"ethereum"`
	Tezos    TezosContracts    `json:"tezos"`
}

TokenRegistry aggregates contracts across different blockchains

func (TokenRegistry) AllContractAddresses

func (tr TokenRegistry) AllContractAddresses() ContractAddresses

AllContractAddresses returns a nested map of all contract addresses by blockchain

func (TokenRegistry) TotalSupply

func (tr TokenRegistry) TotalSupply() int

type TokenUpdateSet

type TokenUpdateSet struct {
	Source            string    `structs:"source,omitempty"`
	AssetID           string    `structs:"assetID,omitempty"`
	Fungible          bool      `structs:"fungible,omitempty"`
	Edition           int64     `structs:"edition,omitempty"`
	EditionName       string    `structs:"editionName,omitempty"`
	ContractAddress   string    `structs:"contractAddress,omitempty"`
	MintedAt          time.Time `structs:"mintedAt,omitempty"`
	LastRefreshedTime time.Time `structs:"lastRefreshedTime"`
	LastActivityTime  time.Time `structs:"lastActivityTime,omitempty"`
}

type TotalBalance

type TotalBalance struct {
	ID    string `bson:"_id"`
	Total int    `bson:"total"`
}

type TransactionDetails

type TransactionDetails struct {
	From      string
	To        string
	IndexID   string
	Timestamp time.Time
}

type URLDownloader

type URLDownloader struct {
	// contains filtered or unexported fields
}

func NewURLDownloader

func NewURLDownloader(url string, timeout time.Duration) *URLDownloader

func (*URLDownloader) Download

func (d *URLDownloader) Download() (io.ReadCloser, string, error)

type VersionedProjectMetadata

type VersionedProjectMetadata struct {
	Origin ProjectMetadata `json:"origin" structs:"origin" bson:"origin"`
	Latest ProjectMetadata `json:"latest" structs:"latest" bson:"latest"`
}

VersionedProjectMetadata is a structure that manages different versions of project metadata. Currently, it maintains two version: the original one and the latest one.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL