crawling 미사용 코드 정리 및 다중 카테고리 태깅 리팩토링 by GulSam00 · Pull Request #194 · GulSam00/singcode

GulSam00 · 2026-04-16T13:25:14Z

Summary

워크플로우에서 사용되지 않는 일회성/구버전 크롤링 스크립트 7개 삭제
다중 카테고리 태깅 함수 getSongFullTag.ts 추가 (language 1개 + genre 1개 + origin 1~3개)

삭제된 파일

src/postScrapeSongs.ts, src/updateJpnSongs.ts
src/crawling/crawlWiki.ts, src/crawling/replaceSupabaseFailed.ts
src/utils/transChatGPT.ts, src/utils/logData.ts, src/utils/namuArgList.ts

Test plan

pnpm lint 통과 확인
기존 워크플로우 5개 정상 동작 확인 (삭제된 파일 미참조)
getSongFullTag.ts 수동 실행 테스트

Closes #193

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

vercel · 2026-04-16T13:25:24Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
singcode	Ready	Preview, Comment	Apr 16, 2026 1:25pm

qodo-code-review · 2026-04-16T13:25:33Z

Review Summary by Qodo

Clean up unused crawling code and add multi-category song tagging

✨ Enhancement 📦 Other

Walkthroughs

Description

• Deleted 7 unused crawling scripts and utilities
  - Removed crawlWiki.ts, replaceSupabaseFailed.ts, postScrapeSongs.ts, updateJpnSongs.ts
  - Removed transChatGPT.ts, logData.ts, namuArgList.ts
• Added multi-category auto-tagging function for songs
  - Supports language (1), genre (1), and origin (1-3) tags
  - Implements LLM-based classification with artist tag reuse optimization

Diagram

flowchart LR
  A["Unused Crawling Scripts"] -->|Delete| B["Cleaned Codebase"]
  C["Song Metadata"] -->|Process| D["getSongFullTag.ts"]
  D -->|AI Classification| E["Language/Genre/Origin Tags"]
  D -->|Reuse Existing| F["Artist Tag Cache"]

File Changes

1. packages/crawling/src/crawling/crawlWiki.ts Cleanup +0/-119

Remove unused Namu Wiki scraping module

• Deleted entire file containing Namu Wiki scraping functions
• Removed scrapeSongs(), scrapeAllSongs(), and scrapeUtaiteSongs() functions
• These functions were no longer used in active workflows

packages/crawling/src/crawling/crawlWiki.ts

2. packages/crawling/src/crawling/replaceSupabaseFailed.ts Cleanup +0/-28

Remove unused Supabase failed data handler
• Deleted entire file with incomplete/commented-out database replacement logic
• Contained mostly commented code for handling failed KY song entries
• No longer referenced in active workflows
packages/crawling/src/crawling/replaceSupabaseFailed.ts

3. packages/crawling/src/postScrapeSongs.ts Cleanup +0/-40

Remove unused song posting script

• Deleted entire file containing song posting functions
• Removed postSongs(), postAllSongs(), and postUtaiteSongs() functions
• All functions were commented out and marked as unusable

packages/crawling/src/postScrapeSongs.ts

View more (5)

4. packages/crawling/src/updateJpnSongs.ts Cleanup +0/-55

Remove unused Japanese song update script

• Deleted entire file for Japanese song translation and database updates
• Removed logic for translating song titles using ChatGPT API
• No longer part of active workflow pipeline

packages/crawling/src/updateJpnSongs.ts

5. packages/crawling/src/utils/getSongFullTag.ts ✨ Enhancement +171/-0

Add multi-category song auto-tagging function

• New file implementing multi-category song auto-tagging system
• Supports language (100-199), genre (200-299), and origin (300-399) tag ranges
• Implements artist tag reuse optimization to reduce API calls
• Uses OpenAI GPT API with JSON response format for reliable tag classification
• Includes Hangul/Kana detection for pre-determining language tags

packages/crawling/src/utils/getSongFullTag.ts

6. packages/crawling/src/utils/logData.ts Cleanup +0/-33

Remove unused file logging utilities

• Deleted entire file containing file-based logging utilities
• Removed functions for loading/saving crawl results and valid KY songs
• No longer needed in refactored workflow

packages/crawling/src/utils/logData.ts

7. packages/crawling/src/utils/namuArgList.ts Cleanup +0/-52

Remove unused Namu Wiki artist arguments

• Deleted entire file containing Namu Wiki artist argument configurations
• Removed JpArgList array with 26 Japanese artist crawling configurations
• Associated with deleted crawlWiki.ts functionality

packages/crawling/src/utils/namuArgList.ts

8. packages/crawling/src/utils/transChatGPT.ts Cleanup +0/-69

Remove unused ChatGPT translation utility

• Deleted entire file with ChatGPT-based Japanese-to-Korean translation logic
• Removed TranslationAssistant class and transChatGPT() function
• Functionality replaced by new multi-category tagging system

packages/crawling/src/utils/transChatGPT.ts

qodo-code-review · 2026-04-16T13:25:34Z

Code Review by Qodo

🐞 Bugs (3) 📘 Rule violations (0) 📎 Requirement gaps (1)

🐞\ ☼ Reliability (1) ⚙ Maintainability (1) ➹ Performance (1)

📎\ ≡ Correctness (1)

1. getExistingArtistTags() returns too many origins 📎 ≡

Description

When reusing an artist’s existing tags, the function returns all origin tags without enforcing the
required 1–3 origin tag limit. This can violate the tagging cardinality requirements and produce
invalid outputs for downstream consumers.

Code

packages/crawling/src/utils/getSongFullTag.ts[R74-81]

+  const language = tagIds.find(id => isValidTagId(id, LANGUAGE_TAG_MIN, LANGUAGE_TAG_MAX));
+  const genre = tagIds.find(id => isValidTagId(id, GENRE_TAG_MIN, GENRE_TAG_MAX));
+  const origin = tagIds.filter(id => isValidTagId(id, ORIGIN_TAG_MIN, ORIGIN_TAG_MAX));
+
+  // 세 카테고리 모두 있어야 재사용
+  if (language && genre && origin.length > 0) {
+    return [language, genre, ...origin];
+  }

Relevance

⭐⭐⭐ High
Reuse path returns all origin tags; can exceed required 1–3 origin limit (LLM path slices but reuse
doesn’t).
ⓘ Recommendations generated based on similar findings in past PRs

Evidence
PR Compliance ID 2 requires origin tag IDs (300–399) to be between 1 and 3 inclusive. The new
implementation collects all origin tags via filter(...) and returns them without slice(0, 3) or
other limiting logic when reusing existing tags.
Implement multi-category song tagging (language/genre/origin) via getSongFullTag.ts
packages/crawling/src/utils/getSongFullTag.ts[74-81]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`getExistingArtistTags()` reuses existing artist tags but can return more than 3 origin tags, violating the required origin cardinality (1–3).

## Issue Context
Compliance requires: language exactly 1 (100–199), genre exactly 1 (200–299), origin 1–3 (300–399). The LLM path already caps origins via `slice(0, 3)`, but the reuse path does not.

## Fix Focus Areas
- packages/crawling/src/utils/getSongFullTag.ts[74-81]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

2. Conflicting autoTagSong APIs 🐞 ⚙

Description

새 파일은 autoTagSong(): Promise<number[] | null>을 export 하는데, 기존 워크플로우는 `autoTagSong():
Promise<number | null>`를 전제로 동작합니다. 동일한 export 이름이 공존해 잘못 import하면 런타임/타입 불일치가 발생하고, 현재 tag-songs
크론은 여전히 단일 태그만 저장해 다중 태깅 리팩토링이 적용되지 않습니다.

Code

packages/crawling/src/utils/getSongFullTag.ts[R147-166]

+export const autoTagSong = async (
+  title: string,
+  artist: string,
+  tagsPrompt: string,
+): Promise<number[] | null> => {
+  try {
+    if (!tagsPrompt) return null;
+
+    // 1단계: 같은 아티스트의 기존 태그가 모두 있으면 재사용 (API 호출 생략)
+    const existingTags = await getExistingArtistTags(artist);
+    if (existingTags) return existingTags;
+
+    // 2단계: 한글/가나 감지로 언어 태그만 사전 결정
+    const titleAndArtist = title + artist;
+    const hasHangul = /[ㄱ-ㅎㅏ-ㅣ가-힣]/.test(titleAndArtist);
+    const hasKana = /[ぁ-んァ-ヶ]/.test(titleAndArtist);
+    const knownLanguageTag = hasHangul ? TAG_KOREAN : hasKana ? TAG_JAPANESE : undefined;
+
+    // 3단계: LLM으로 genre + origin (+ 필요 시 language) 판별
+    return await getTagsFromLLM(title, artist, tagsPrompt, knownLanguageTag);

Relevance

⭐⭐ Medium
Potential API/type mismatch if old single-tag autoTagSong is still used, but repo usage not
verifiable here.
ⓘ Recommendations generated based on similar findings in past PRs

Evidence
getSongFullTag.ts의 autoTagSong은 number[]를 반환하도록 정의되어 있습니다. 하지만 현재 tag-songs 크론은 getSongTag.ts에서
autoTagSong을 import하고 단일 tagId를 [tagId]로 감싸 insert 하므로, 새 다중 태깅 유틸이 실제 플로우에 연결되지 않았고 이름 충돌로 오용 위험이
큽니다.
packages/crawling/src/utils/getSongFullTag.ts[147-166]
packages/crawling/src/cron/taggingSongs.ts[1-4]
packages/crawling/src/cron/taggingSongs.ts[29-41]
packages/crawling/src/utils/getSongTag.ts[48-53]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
다중 태깅용 `autoTagSong()`(number[] 반환)과 기존 단일 태깅용 `autoTagSong()`(number 반환)이 공존하며 export 이름이 동일합니다. 현재 크론은 단일 태그 삽입만 수행하므로 다중 태깅 리팩토링이 실제로 적용되지 않고, 추후 import 실수 시 타입/런타임 불일치가 발생합니다.

## Issue Context
`postSongTagsDB(songId, tagIds)`는 원래부터 다중 tagIds 삽입을 지원합니다.

## Fix Focus Areas
- packages/crawling/src/utils/getSongFullTag.ts[147-166]
- packages/crawling/src/cron/taggingSongs.ts[1-4]
- packages/crawling/src/cron/taggingSongs.ts[29-41]

## Suggested change
- 크론이 다중 태깅을 의도한다면 import를 `getSongFullTag`로 변경하고, 반환된 `number[]`를 그대로 `postSongTagsDB(song.id, tagIds)`에 전달하세요.
- 다중 태깅을 아직 사용하지 않을 계획이라면, 새 모듈의 export 이름을 명확히 구분(예: `autoTagSongFull`)하거나 기존 `getSongTag.ts`를 대체/삭제해 API 충돌 가능성을 제거하세요.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

3. Unfiltered tags prompt 🐞 ➹

Description

getTagsForPrompt()가 주석상 100~399 태그를 가져온다고 되어 있지만 실제로는 tags 테이블 전체를 필터 없이 조회합니다. 그 결과 LLM 프롬프트가 불필요하게
커지고, 허용 범위 밖 태그가 포함되어 LLM 결과가 검증에서 reject되어 null로 떨어질 가능성이 커집니다.

Code

packages/crawling/src/utils/getSongFullTag.ts[R40-46]

+export const getTagsForPrompt = async (): Promise<string> => {
+  const supabase = getClient();
+  const { data: tags, error } = await supabase
+    .from('tags')
+    .select('id, name, category')
+    .order('id');
+

Relevance

⭐⭐⭐ High
Comment promises tags 100~399 but query selects entire tags table; prompt bloat/invalid tags risk is
real.
ⓘ Recommendations generated based on similar findings in past PRs

Evidence
getSongFullTag.ts는 100~399 범위를 사용하도록 상수(LANGUAGE/GENRE/ORIGIN min/max)를 정의하고, LLM 결과도 해당 범위로 검증합니다.
하지만 getTagsForPrompt()의 Supabase 쿼리는 id 범위 조건이 없어 tags 전체가 프롬프트에 포함됩니다.
packages/crawling/src/utils/getSongFullTag.ts[12-18]
packages/crawling/src/utils/getSongFullTag.ts[37-46]
packages/crawling/src/utils/getSongFullTag.ts[137-145]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`getTagsForPrompt()`가 tags 테이블을 전체 조회하여 프롬프트에 불필요한 태그까지 포함됩니다. 주석/의도(100~399)와 구현이 불일치하며, 토큰/비용 증가 및 LLM 결과가 범위 검증에서 실패(null)할 확률을 높입니다.

## Issue Context
이 모듈은 language(100~199), genre(200~299), origin(300~399)만 사용하며, LLM 결과도 해당 범위로 검증합니다.

## Fix Focus Areas
- packages/crawling/src/utils/getSongFullTag.ts[37-46]

## Suggested change
Supabase 쿼리에 `.gte('id', LANGUAGE_TAG_MIN).lt('id', ORIGIN_TAG_MAX)`(또는 동등한 조건) 필터를 추가해 100~399만 프롬프트에 포함되도록 수정하세요.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

4. Nondeterministic tag reuse 🐞 ☼

Description

getExistingArtistTags()가 .limit(1)만 사용하고 정렬이 없어 아티스트의 어떤 곡이 선택될지 비결정적이며, Supabase error를 확인/로그하지
않습니다. 이로 인해 동일 아티스트에 대해 재사용 결과가 일관되지 않거나, DB 조회 실패가 조용히 LLM 호출로 전환되어 비용/결과가 흔들립니다.

Code

packages/crawling/src/utils/getSongFullTag.ts[R60-69]

+const getExistingArtistTags = async (artist: string) => {
+  const supabase = getClient();
+  const { data } = await supabase
+    .from('songs')
+    .select('song_tags!inner(tag_id)')
+    .eq('artist', artist)
+    .gte('song_tags.tag_id', LANGUAGE_TAG_MIN)
+    .lt('song_tags.tag_id', ORIGIN_TAG_MAX)
+    .limit(1);
+

Relevance

⭐⭐⭐ High
.limit(1) without order is nondeterministic and { data } ignores Supabase error, hiding DB
failures.
ⓘ Recommendations generated based on similar findings in past PRs

Evidence
해당 쿼리는 order 없이 limit(1)이라 임의의 songs row를 가져올 수 있고, { data }만 받아 error를 무시합니다. 반면 동일 패키지의 다른 DB
헬퍼들은 error 발생 시 throw로 실패를 명확히 합니다.
packages/crawling/src/utils/getSongFullTag.ts[60-69]
packages/crawling/src/supabase/getDB.ts[88-99]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`getExistingArtistTags()`의 기존 태그 재사용 조회가 비결정적이고(정렬 없음) DB 에러를 무시하여, 재사용 최적화가 불안정해집니다.

## Issue Context
같은 아티스트에 대해 여러 곡이 존재할 수 있으며, 재사용은 비용 절감 경로입니다.

## Fix Focus Areas
- packages/crawling/src/utils/getSongFullTag.ts[60-69]

## Suggested change
- Supabase 응답에서 `{ data, error }`를 받아 `error`를 로그/처리하세요.
- `.order('updated_at', { ascending: false })` 등 명시적 정렬을 추가해 선택을 결정적으로 만드세요.
- (선택) 여러 row를 가져와(language/genre/origin 모두 있는) row를 고르는 방식으로 재사용 성공률을 높이세요.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

ⓘ The new review experience is currently in Beta. Learn more

qodo-code-review · 2026-04-16T13:30:20Z

+  const language = tagIds.find(id => isValidTagId(id, LANGUAGE_TAG_MIN, LANGUAGE_TAG_MAX));
+  const genre = tagIds.find(id => isValidTagId(id, GENRE_TAG_MIN, GENRE_TAG_MAX));
+  const origin = tagIds.filter(id => isValidTagId(id, ORIGIN_TAG_MIN, ORIGIN_TAG_MAX));
+
+  // 세 카테고리 모두 있어야 재사용
+  if (language && genre && origin.length > 0) {
+    return [language, genre, ...origin];
+  }


1. getexistingartisttags() returns too many origins 📎 Requirement gap ≡ Correctness

When reusing an artist’s existing tags, the function returns all origin tags without enforcing the required 1–3 origin tag limit. This can violate the tagging cardinality requirements and produce invalid outputs for downstream consumers.

Agent Prompt

## Issue description `getExistingArtistTags()` reuses existing artist tags but can return more than 3 origin tags, violating the required origin cardinality (1–3). ## Issue Context Compliance requires: language exactly 1 (100–199), genre exactly 1 (200–299), origin 1–3 (300–399). The LLM path already caps origins via `slice(0, 3)`, but the reuse path does not. ## Fix Focus Areas - packages/crawling/src/utils/getSongFullTag.ts[74-81]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

qodo-code-review · 2026-04-16T13:30:20Z

+export const autoTagSong = async (
+  title: string,
+  artist: string,
+  tagsPrompt: string,
+): Promise<number[] | null> => {
+  try {
+    if (!tagsPrompt) return null;
+
+    // 1단계: 같은 아티스트의 기존 태그가 모두 있으면 재사용 (API 호출 생략)
+    const existingTags = await getExistingArtistTags(artist);
+    if (existingTags) return existingTags;
+
+    // 2단계: 한글/가나 감지로 언어 태그만 사전 결정
+    const titleAndArtist = title + artist;
+    const hasHangul = /[ㄱ-ㅎㅏ-ㅣ가-힣]/.test(titleAndArtist);
+    const hasKana = /[ぁ-んァ-ヶ]/.test(titleAndArtist);
+    const knownLanguageTag = hasHangul ? TAG_KOREAN : hasKana ? TAG_JAPANESE : undefined;
+
+    // 3단계: LLM으로 genre + origin (+ 필요 시 language) 판별
+    return await getTagsFromLLM(title, artist, tagsPrompt, knownLanguageTag);


2. Conflicting autotagsong apis 🐞 Bug ⚙ Maintainability

새 파일은 autoTagSong(): Promise<number[] | null>을 export 하는데, 기존 워크플로우는 `autoTagSong(): Promise<number | null>`를 전제로 동작합니다. 동일한 export 이름이 공존해 잘못 import하면 런타임/타입 불일치가 발생하고, 현재 tag-songs 크론은 여전히 단일 태그만 저장해 다중 태깅 리팩토링이 적용되지 않습니다.

Agent Prompt

## Issue description 다중 태깅용 `autoTagSong()`(number[] 반환)과 기존 단일 태깅용 `autoTagSong()`(number 반환)이 공존하며 export 이름이 동일합니다. 현재 크론은 단일 태그 삽입만 수행하므로 다중 태깅 리팩토링이 실제로 적용되지 않고, 추후 import 실수 시 타입/런타임 불일치가 발생합니다. ## Issue Context `postSongTagsDB(songId, tagIds)`는 원래부터 다중 tagIds 삽입을 지원합니다. ## Fix Focus Areas - packages/crawling/src/utils/getSongFullTag.ts[147-166] - packages/crawling/src/cron/taggingSongs.ts[1-4] - packages/crawling/src/cron/taggingSongs.ts[29-41] ## Suggested change - 크론이 다중 태깅을 의도한다면 import를 `getSongFullTag`로 변경하고, 반환된 `number[]`를 그대로 `postSongTagsDB(song.id, tagIds)`에 전달하세요. - 다중 태깅을 아직 사용하지 않을 계획이라면, 새 모듈의 export 이름을 명확히 구분(예: `autoTagSongFull`)하거나 기존 `getSongTag.ts`를 대체/삭제해 API 충돌 가능성을 제거하세요.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

GulSam00 and others added 2 commits April 16, 2026 22:24

chore : 워크플로우 미사용 크롤링 스크립트 삭제 (#193)

6a52ef2

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

feat : 다중 카테고리 태깅 함수 추가 (language/genre/origin) (#193)

005a55e

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

vercel Bot deployed to Preview April 16, 2026 13:25 View deployment

GulSam00 merged commit d51605a into develop Apr 16, 2026
2 checks passed

GulSam00 deleted the refactor/193-cleanupCrawlingAndMultiTag branch April 16, 2026 13:26

qodo-code-review Bot reviewed Apr 16, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

crawling 미사용 코드 정리 및 다중 카테고리 태깅 리팩토링#194

crawling 미사용 코드 정리 및 다중 카테고리 태깅 리팩토링#194
GulSam00 merged 2 commits into
developfrom
refactor/193-cleanupCrawlingAndMultiTag

GulSam00 commented Apr 16, 2026

Uh oh!

vercel Bot commented Apr 16, 2026 •

edited

Loading

Uh oh!

qodo-code-review Bot commented Apr 16, 2026

Uh oh!

qodo-code-review Bot commented Apr 16, 2026 •

edited

Loading

Uh oh!

Uh oh!

qodo-code-review Bot Apr 16, 2026

Uh oh!

qodo-code-review Bot Apr 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

GulSam00 commented Apr 16, 2026

Summary

삭제된 파일

Test plan

Uh oh!

vercel Bot commented Apr 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

qodo-code-review Bot commented Apr 16, 2026

Review Summary by Qodo

Walkthroughs

File Changes

Uh oh!

qodo-code-review Bot commented Apr 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Review by Qodo

Uh oh!

Uh oh!

qodo-code-review Bot Apr 16, 2026

Choose a reason for hiding this comment

Uh oh!

qodo-code-review Bot Apr 16, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

vercel Bot commented Apr 16, 2026 •

edited

Loading

qodo-code-review Bot commented Apr 16, 2026 •

edited

Loading